DEV Community: Talina

3 Integration Testing Tips For Developers To Release Software With Confidence

Talina — Thu, 20 Jan 2022 04:40:49 +0000

Earlier, we discussed the thumb rules of unit testing.

Unit tests form the first level in the Test Pyramid - we achieve faster results by assuming behaviour for all external boundaries.

Integration tests are a step ahead - they sufficiently test the communication over these external boundaries.

Here are 3 things developers should keep in mind while writing integration tests.

#1 Deconstruct The External Interactions

Allow integration tests to be narrow.

Integration tests validate if two separately developed units work well together.

We can allow integration testing to be white-box if we test each integration point at a time.

Example: An application interacts with a database and an external API service.

Integration testing for database interaction -

Start a database.
Invoke the code that performs database operations.
Validate the operations were successful.

The integration test can mock all other external interactions as it only tests database interactions.

Narrow integration tests test for one integration point at a time.

#2 Use Local Environment

We don't need to initiate an entire stack to run integration tests if they are narrow.

All external services should be set up locally so that integration tests can be run during the development phase and reduce any operational hassles.

A local setup makes integration tests fast and easy to run - they can be added to the build pipeline with ease.

Example: A single deployment file that spins up services on your laptop using an orchestrator such as "docker".

A local setup makes integration tests more deterministic - we remove network failures.

#3 Test Doubles Are Acceptable

External services don't always imitate exact behaviour in a local environment.

The local instance could be unofficial or mismatch the latest versioned behaviour.

In such a case, we should opt for mocking the service at the last level of interaction by using appropriate test doubles.

These test doubles guarantee that the public interfaces work as expected.

Narrow integration tests validate clients not servers.

Top 3 Ways to Log Errors to Simplify Debugging For Systems Engineers

Talina — Sat, 15 Jan 2022 15:22:21 +0000

How often do you find yourself spending hours trying to triage a failure in production?

Debugging is challenging because we don't know what we're looking for, and to top it off, we have no way to pick on the trail of errors.

The most obvious first step is to ensure all services are logging errors to a standard logging tool.

Logs are your guardian angels when it comes to finding root causes of failures.

Let's understand how to befriend these angels and streamline debugging.

Here's a sample scenario.

A user wants to open an online bank account.

The backend function calls are -

1. (api) POST /accounts
    2. (api) CreateAccountHandler()
       2.1 (internal) AccountsSvc.InitiateCreateAccount()
       2.2 (internal) AccountsSvc.ValidateUser()
           2.2.1 (validator) ValidateID()
       2.3 (internal) AccountsSvc.ValidateBankDetails()
           2.3.1 (validator) ValidateID()
       2.4 (internal) AccountsSvc.CreateAccount()
           2.4.1 (database) Store.SaveAccountDetails()
                 2.4.1.1 (writer) Data.Write()
     3. (serialization) Response.Serialize()

#1 Relative Messaging

Steps 2.2.1 and 2.3.1 have the same ValidateID() function call.
If the ValidateID() call fails to validate the user, we want to log about this correlation to pinpoint the exact failure.
Use this template across all appropriate levels -

Failed to perform X while attempting to do Y

So, a failure in ValidateID() at 2.2.1 would result in errors logged from the bottommost to the topmost caller -

Error Log Trace
- (validator/ValidateID) Failed to validate the ID.
- (internal/ValidateUser) Failed to validate the ID while attempting to validate user details.
- (internal/InitiateCreateAccount) Failed to validate user details while attempting to create a new account.

#2 Log Across Microservices

Imagine there are 2 microservices involved in this sample example.

API service accepts incoming http requests.
Internal Service performs the CreateAccount operations.

If all microservice log the shared context, correlating the API service errors with internal service becomes easy.

The shared context is metadata that helps filter loglines - a request ID, a user ID or an account ID.

A logline is the "vertex", and the shared context is the segment that connects each vertex.

#3 Avoid Pseudo Errors

We retry operations to deal with intermittent failures such as a flaky network.
Use the correct log level to separate intermittent failures from final failures.

A simple way to do this is by using the "warn" level for intermittent failures and the "error" level for the final failure.

I hope these simple improvements streamline debugging failures.

Want to add more best practices to this list? Let me know in the comments!

Let's Chat About Contexts in Go

Talina — Thu, 13 Jan 2022 17:45:33 +0000

Today I want to have a brief chat about Contexts in Go. :)

Every Go developer uses Contexts in their coding lifetime.

Read this profound yet straightforward blog if you are new to Go and Contexts.

To excel as a developer, we should actively make an effort to understand the "why".

There are many great articles on the web about using Contexts in Go and the how-to.

But, let's mull over why should Contexts be used in the first place.

Carriers of Metadata

Contexts carry miscellaneous metadata that is helpful for each invoked function call. This metadata could include the caller's information or general identification information.

This metadata is also helpful while debugging incoming requests if we log the messages with the context information.

Resources Saver!

Contexts offer an ability to cancel an ongoing operation if -

The caller invoked a cancellation -
- Imagine a scene where a flick to the first domino makes all subsequent dominos fall.
- That is how the parent Context's CancelFunc cancels derived contexts.
When the operation takes longer than expected -
- In this case, the Context's cancel logic is invoked when the configured deadline exceeds, and this cancellation propagates to the derived contexts.

The advantage of context cancellation is that all goroutines exit immediately, and the application can reclaim used resources such as memory.

Using Contexts is a way to safeguard resource consumption.

ADRs Done Right

Talina — Wed, 12 Jan 2022 18:22:45 +0000

As a Software Engineer, I have had the opportunity to write multiple Architecture Decision Records(ADRs).

Head over to Spotify's Blog Post for in-depth reading.

An ADR answers two questions -

What is the problem?
How should we solve it?

Writing ADRs brings clarity and binds teams to a common goal.

ADRs serve as suitable communication mediums for new team members - they can live the past without being there.

Let's dive deeper into the art of writing an ADR.

Depth of Solution

ADRs should solve the problem, but the way it's solved shouldn't matter if the constraints are apparent.

Ex: Rotate credentials every 24 hours.

The implementation doesn't matter - it could be a cron, manual or a scheduler service; the details lie with the folks who work on it.

What is necessary is that the credentials should have a shelf-life of 24 hours.

State Your Assumptions

State assumptions to gauge if the design becomes obsolete.

Ex: How much data traffic rate can the software support?

Provided the technique is reliable and resilient, we should -

Architect for the future, but settle for simplicity.

Security Aspects

Security is vital for software dealing with sensitive customer information, including accessing customer data.

A "Threat Modeling" exercise done in advance to cover all security breaches goes a long way.

Costing Aspects

Every offering of your software product has a fixed, affordable budget, and a technical decision is almost always strongly linked with this cost.

Does the design involve a managed service? It's good to include baseline costing for data transfer, hourly rates and so on.

Single Decision Record

Before writing the ADR, an array of discussions happen in the background.

Every ADR should have a "decision" section which states a single locked-in decision - assimilation of all the discussion meetings.

In conclusion, when it comes to writing ADRs -

The key is to promote transparency and participation while maintaining focus and ownership of the architecture.

That's all for today! Until later!

Thumb Rules of Unit Testing

Talina — Tue, 11 Jan 2022 16:45:07 +0000

My ideas on writing meaningful unit tests were crystallized when I read "The Practical Test Pyramid" today.

As a novice programmer, I went in mindlessly, not knowing how to test my code.

The Practical Test Pyramid is an excellent guide, but I want to emphasize unit testing today.

Why Write Unit Tests?

As the name suggests - you test a "unit" of your code.

Unit tests are beneficial in ensuring the code works as intended - it is the first step towards being confident about the logic.

Unit tests have the narrowest scope of all the tests in your test suite.

A service accesses a database to perform a set of computations; the unit tests will cover all edge cases for the performed computations. The access to the database is "assumed".

How to Write Good Unit Tests?

Cover Non-Trivial Cases

Test for all edge cases in the code.

Typically, you won't test if input deserialization works unless you've written the library itself.

When it comes to unit testing, aim for confidence coverage and not code coverage.

Test for Observable Behavior

Loosely couple tests and code. Refactoring code is a common practice, but refactoring unit tests is stressful.

Unit tests should always answer two questions -

What is the input?
Is the output expected?

Structure

Tests, like your code, should be easy to read and understand.

Most supporters of TDD derive code from tests. Hence, a readable structure goes a long way in translating behaviour into code.

Use "Given, When, Then"

Given an input X
When function Y executes
Then the outcome is Z

Ensure you maintain a delicate balance between DRY(Don't Repeat Yourself) and DAMP(Descriptive and Meaningful Phrases).

TestDoubles

We can use various TestDoubles to replace production objects for testing purposes.

Use stubs in cases where branching logic doesn't matter. We don't care if the database write fails.
Use mocks when the order and the output matter. We care if the database write fails the second time.

More at Mocks aren't Stubs.

Let me know how you write unit tests in the comments!

5 Big Mistakes Companies Make When Testing Software

Talina — Tue, 11 Jan 2022 04:40:28 +0000

Most software companies struggle with releasing quality code that doesn't break critical user workflows.

Companies invest significant dollars in testing their product to counter this but often ship a P0 bug anyway.

Here are 5 root causes for software bugs.

#1 Inconsistent Testing Environments

A common mistake and often a harder one to fix during the nascent phases of software.

Naturally, if the pre-production environment never hits an edge case that can occur in production, it is impossible to validate enhancements accurately.

Thought: Deploy a snapshot of the production environment as a pre-release environment.

#2 Backwards Incompatibility

Engineering teams write code to work in the "present" and the "future", making version testing hard. Software should not be forward-facing.

Example: Release #1 deploys software with a v1 dependency, and release #2 now uses a v2 version of the dependency. This change is non-idempotent - we wipe out a variable value by storing only the latest state.

What happens to the deployed state that works with a v1 dependency? Does it break or work with the new release?

Thought: Always support X states of existing data. Streamline rollbacks.

#3 Treating Infrastructure and Microservices Separately

As more folks campaign for infrastructure as code, we move towards automating infrastructure deployments.

But does a change in automation break existing infrastructure?

Thought: Can backward compatibility testing solve this problem?

#4 Ignoring the Network

Assuming the network behaves the same irrespective of time and region is a recipe for software failure.

Thought: Can Edge Testing gauge pre-release software inconsistencies?

#5 Testing Features in Silos

With the microservices architecture, a change in one service can cause a failure in unrelated user functionality. Testing a new release in a silo for a fixed subset of features doesn't help.

Thought: Replay tests for all critical user workflows for every release to generate an accurate failures heatmap.

Let me know if you've solved any of these mistakes and how.

Until later! :)

Kickstart Your Journey as a Go Engineer in 4 Easy Steps

Talina — Sun, 09 Jan 2022 18:35:33 +0000

A structured approach to learning goes a long way in maximizing your output as a developer. Why struggle with theory when you can code immediately?

As an entrant, look for small wins every day - the key is to get instant gratification by writing production-grade Go.

#1 Win at the Syntax Game

A developer communicates through code; the first obvious step is to learn the syntax - a language's grammar.

Learning syntax can be slow - one has to remember many minute details. But thanks to gobyexample, this has been made simple.

Run the code snippets in Go's Playground. Tweak the code and implement various use-cases.

Another excellent news: Go offers limited in-built data structures, so the learning curve is shallow.

#2 Read Code to Write Code

Read code to incorporate different design patterns.

Read code with an eye for design but a vision to improve it.

Review code - read other reviewers' comments, ask questions, seek clarity. This is a proven method to grasp the language better.

#3 Master Best Practices

Want to master a coding language? Learn its best practices.

One language's guidelines don't apply to another. Even as a proficient Gopher, you'll find yourself revisiting Effective Go.

Conquering best practices goes a long way in writing maintainable code that can be tested quickly and refactored fewer times.

Communicating with your code will be easier if your code has a structure that most engineers know.

#4 Make Your First Contribution!

Make your first contribution while learning the syntax.

Create an HTTP service. Next, are you able to write a CLI service in Go?

Shadow other developers. Pick a low-hanging fruit - fixing a bug in a codebase, enhancing an API or writing a new script. Find a proficient colleague who can review your work.

There are tons of open-source projects in Go; contribute to these fantastic projects.

I hope you master Go by following these quick tips.
May the force be with you!