DEV Community: Jaap Groeneveld

Software Testing

Jaap Groeneveld — Sat, 10 Dec 2022 19:47:34 +0000

I think that software testing is essential for ensuring the quality and reliability of software. I believe that a comprehensive testing strategy, including a mix of different types of testing, is necessary to identify and fix any issues with the software before it is released to users. I also believe that testing should be an ongoing process, with tests being performed regularly throughout the development cycle to catch problems early on and prevent them from becoming bigger issues later. Additionally, I believe that testing should be an integral part of the development process, with developers and testers working together to ensure that the software meets the specified requirements and works as intended.

In this article about testing software I want to explain what software testing is and elaborate on my philosphy.

What is software testing and why is it important

Software testing is the process of evaluating a software application or system to determine whether it meets the specified requirements and works as intended. It is an important part of the software development process because it helps to identify bugs, defects, and other issues with the software, and ensures that the final product is of high quality and fit for its intended purpose.

Software can be tested manually and also automatically. The two are not mutually exclusive. Automated testing is not a replacement for manual testing, but it is a useful complement to manual testing. Automated tests can help to quickly and efficiently identify problems with the software, while manual testing can be used to explore the software and test it in ways that automated tests may not cover.

What is manual testing?

In manual testing the tester manually performs a series of actions on the software, such as clicking on buttons and entering data, to test its functionality and identify any bugs or defects.
This is in contrast to automated testing, in which the testing is performed using specialized software tools that automatically execute the test cases. Manual testing is useful for testing the user interface and other aspects of the software that may be difficult to test using automated tools. It is also useful for exploratory testing, in which the tester has the freedom to try out different scenarios and test the software in ways that may not be covered by the pre-defined test cases.

A lot of companies employ specific QA (Quality Assurance or Quality Assistance) experts that help with manual testing as well as with automated testing.

What is automated testing?

Automated testing is a way of testing software using special tools that can run tests automatically without human intervention. These tools can run tests quickly and accurately, and they can be used to test parts of the software that are difficult to test manually. Automated testing is useful because it can help to identify problems with the software and ensure that it works as intended.

Automated tests can be run quickly and repeatedly, which is useful for performing regression testing to ensure that changes to the software haven't introduced new bugs.
Automated tests can be run without human intervention, which is useful for running tests overnight or in continuous integration environments.
Automated tests are (or should be) consistent and accurate, since they are performed using the same set of instructions every time.
Automated tests can cover a larger number of test cases, since they don't rely on a human tester to manually perform each test.
Automated tests can be used to test parts of the software that are difficult or impossible to test manually, such as performance or security.

There are many different types of testing, but some common ones include:

Unit testing: This involves testing individual components or units of the software to ensure they function as intended.
Integration testing: This involves testing the integration of different components or units of the software to ensure they work together as expected.
System / End-to-end testing: This involves testing the entire system as a whole to ensure it meets the specified requirements and works as intended.
Regression testing: This involves testing the software after making changes to ensure that the changes have not introduced new bugs or defects.
Acceptance testing: This involves testing the software from the end user's perspective to ensure that it is usable and meets their needs.
Performance testing: This involves testing the software to ensure it performs well under expected workloads and doesn't crash or slow down.
Security testing: This involves testing the software to ensure it is secure and protects sensitive data from unauthorized access.
Smoke testing: This involves testing the software to ensure that the most important functions of a software application or system are working properly in a live environment.

The test pyramid

The test pyramid is a concept in software testing that suggests that the majority of tests should be unit tests, which are the lowest-level tests, followed by a smaller number of integration tests, and even fewer high-level end-to-end tests. This concept is called the "test pyramid" because it is visualized as a pyramid, with the unit tests forming the base and the end-to-end tests forming the tip of the pyramid.

The idea behind the test pyramid is that unit tests are fast and cheap to write and maintain, and they provide good coverage of the code. Integration tests are a bit more expensive to write and maintain, but they test the integration of different components or units of the software, which is important for ensuring that the software works as intended. End-to-end tests, on the other hand, are the most expensive to write and maintain, but they provide the highest level of confidence that the software works as intended from the end user's perspective. The test pyramid is a general guideline, and the specific mix of tests will depend on the specific software being developed and the goals of the testing.

There are different visual models representing the distribution of tests in the system like the test cone, the test iceberg, the test ladder and the test mosaic. If you are interested, you can do further research.

Not always a 100% test pyramid distribution is the right way to go. The specific approach to testing will depend on the specific software being developed and the goals of the testing. The test pyramid is a useful starting point, but it may not be the best approach for every situation.

The danger of tests

Unit tests are testing individual components (units) of the software to ensure that they function as intended.
A unit is the smallest testable part of the software, such as a function, and a unit test is a test that exercises that unit and verifies its behavior. The idea behind unit testing is to test each unit of the software in isolation, without the need to set up a complex test environment or coordinate the execution of multiple tests. This makes unit tests fast and easy to write and maintain, and they can provide good coverage of the code.

Integration tests are testing the integration of different components or units of the software to ensure that they work together as expected.

The problem with the test pyramid is that a lot of unit tests in the real world are actually not testing behavior nor integration. I am sure you have encounted mock heavy test suites that basically test if a method calls 3 other functions but nothing else is tested. They do not guarantee actual business logic nor do they guarantee actual integration with other parts of the application.

While these tests can still be helpful, they also have some disadvantages, including:

They do not actually test any behavior. There is no real business value in testing that function A calls function B.
Thy can be brittle, meaning that they can break easily if the software is changed. This is because like integration tests, mock heavy unit tests often test the interaction between multiple components, and any changes to the interface of one of those components or the structure of the program can affect the behavior of the test.
The may not provide complete coverage of the software being tested. They may not exercise all the individual components or units of the software in the same way that real unit and integration tests do.

Overall, unit and integration tests are an important part of the software testing process, but they should be used in combination with other types of testing, such as end-to-end tests, to provide the most comprehensive coverage of the software being tested. In addition, mocks should be the last resort not the first tool in the box. Often, it is a good idea to rethink what a unit boundary is and opt for more integration tests rather than test every single method with a lot of mocks.

100% Code Coverage

Code coverage is a measure of how much of your source code is executed when you run your tests. It helps you determine how thorough your tests are, and can help you identify areas of your code that are not being exercised by your tests. This is important because untested code is potentially more likely to contain bugs. Often teams try to aim for 100% code coverage.

Achieving 100% code coverage does not necessarily mean that your code is free of bugs or that it is adequately tested. It just means that your tests are exercising all of the lines of code in your source code. However, it is possible to have tests that cover all lines of code but still do not adequately test the functionality of your code. For example, you might have tests that only call a function with trivial input values, which means that the function is not being tested under more realistic or challenging conditions. In this case, even though you have 100% code coverage, your tests are not providing a sufficient level of assurance that your code is working correctly.

Top-Down and Bottom-Up testing

Top-down testing and bottom-up testing are both software testing approaches that are used to test the functionality of a system.

In top-down testing, tests are written at the highest level of the system and then gradually move down to lower levels. This approach starts by testing the overall functionality of the system and then drills down into the individual components and their interactions. The goal of top-down testing is to test the system from the top level and ensure that all of the components work together as intended.

In bottom-up testing, also known as outside-in testing, tests are written at the lowest level of the system first, and then gradually move up to higher levels. This approach starts by testing the individual components of the system and then moves up to test the interactions between these components. The goal of bottom-up testing is to test the system from the perspective of the user or client, focusing on the interactions between the different components of the system.

Both top-down and bottom-up testing have their own strengths and weaknesses. Top-down testing can be useful for identifying defects at the higher levels of the system, but it may not adequately test the interactions between the individual components. Bottom-up testing, on the other hand, can be effective for identifying defects in the interactions between the different components of the system, but it may not adequately test the overall functionality of the system. It is often best to use a combination of both approaches in order to get the most thorough and effective testing.

TDD - Test Driven Design

Test-driven design (TDD) is an approach in which tests are written for a piece of code before the code itself is written. The tests are initially written to fail, and then the code is written to make the tests pass. This approach helps to ensure that the code is written to satisfy the requirements of the tests, and that it is properly tested and working as expected.

In TDD, the development process typically follows these steps:

Write a test that defines a piece of functionality that you want to add to your code.
Run the test and confirm that it fails, because the functionality does not yet exist in the code.
Write the code to make the test pass.
Run the test again and confirm that it passes.
Refactor the code to improve its design and maintainability, without changing its functionality.
Repeat these steps for each piece of functionality that you want to add to your code.

TDD can help to ensure that your code is well-designed, properly tested, and free of defects. It can also help to make your code more modular and easier to maintain.

One common criticism of TDD is that it can be time-consuming and may not always be the most efficient way to develop software. Writing tests for every piece of code that you write can take extra time and effort, and it may not always be clear what tests should be written or how to write effective tests. This can lead to tests that are too broad or too specific, which can make the development process more difficult and less efficient.

Another criticism of TDD is that it can lead to over-testing, where too many tests are written for a given piece of code. This can make the test suite unnecessarily large and complex, and can make it more difficult to maintain and update the tests as the codebase changes over time.

Additionally, some critics argue that TDD can lead to a focus on testing at the expense of other important aspects of software development, such as design and architecture. This can result in code that is well-tested but not well-designed, which can make the code more difficult to maintain and evolve over time.

Overall, while TDD can be a useful tool for ensuring that code is properly tested and working as expected, it is important to use it in the right circumstances and to balance it with other important considerations in software development.

Writing code like writing books

Jaap Groeneveld — Thu, 08 Dec 2022 18:00:00 +0000

Writing readable code is important for a few reasons. First, readable code is easier to understand, which means that it is easier to modify and maintain. This can save time and effort, especially for large projects that may be worked on by multiple people. Second, readable code is more likely to be correct, because it is easier to spot mistakes when the code is easy to understand. Finally, writing readable code can improve your own understanding of the code, because the process of writing clearly and concisely can help you to think more deeply about the problem you are trying to solve.

It is difficult to say exactly how much time developers spend reading and understanding code compared to writing it, as this can vary depending on many factors. However, it is generally accepted that a significant amount of time is spent reading and understanding code, often more than is spent writing it. This is because writing code is just one part of the development process, and it is often necessary to read and understand existing code in order to modify it or add new features. Additionally, many developers find that reading and understanding code can be a challenging and time-consuming task, especially for large and complex projects.

Writing code like books

A good way to make code more readable is to get influenced by how people structure books and articles, especially in technical writing. Usually, the main goal of them is to make a complicated matter understandable by the reader. As this is also one of our goals, lets talk about this.

One way is to use comments and white space to break your code into logical sections, similar to how a book is divided into chapters and paragraphs. You can also use indentation to show the hierarchy and structure of your code, and to make it easier to see how different parts of your code relate to each other. Finally, you can use documentation to provide a high-level overview of your code, similar to how a book might have a table of contents and an introduction.

We want to read code as a set of "paragraphs" each describing the current level of abstraction and referencing down. This can make it easier to understand the logic and flow of the program.

Place functions in the downward direction.
Declare variables close to their usage.
Keep lines short.
Use white space to associate related things and disassociate weakly related.
Think about cohesion

Placing functions in the downward direction can help to show the order in which they are called, and declaring variables close to their usage can make it easier to see how they are used in the code. Keeping lines short can also make the code easier to read and understand, as long as the code is still clear and easy to follow. Long lines are fine if it is obvious what is going on and it is not hiding relevant calls.

Using white space to associate related things and disassociate weakly related items is also a good idea, as it can help to clarify the structure and organization of the code. This can be achieved by using indentation, blank lines, and other techniques to visually separate different parts of the code.

Cohesion is another important concept in writing readable code. This refers to the degree to which the different parts of a module or function work together to achieve a single, well-defined purpose. A high degree of cohesion can make your code easier to understand and maintain, because it helps to ensure that each module or function has a clear and focused role.

In this article I will explain the Step-Down-Rule and Line-of-Sight. To principles that will help you to structure code and make it more readable.

The Step-Down-Rule

The step-down rule is a software engineering principle that suggests that functions should be organized in a hierarchical manner, with higher-level functions calling lower-level functions. This means that higher-level functions should do the overall work of the program, while lower-level functions should perform more specific tasks that support the work of the higher-level functions.

The step-down rule is based on the idea that it is easier to understand and maintain code when it is organized in a clear and logical manner. By organizing your code in a hierarchical fashion, you can make it easier to see the relationship between different parts of the code, and to understand how they work together to achieve the desired outcome. Additionally, following the step-down rule can help to ensure that your code is modular and reusable, which can make it easier to modify and extend over time.

package api

func CreatePost(w http.ResponseWriter, r *http.Request) {
    post, err := getPostFromRequest(r)
    if err != nil {...}

    err = validatePost(post)
    if err != nil { ... }

    err = savePost(post)
    if err != nil { ... }

    http.WriteHeader(201)
}

func getPostFromRequest(r *http.Request) (Post, error) { ... }
func validatePost(post Post) error { ... }
func savePost(post Post) error { ... }

In this code example we have the CreatePost function that outlines the process and deals with the interface to the outside world. It delegates sub-tasks to other functions and only deals with results and errors. This allows you to quickly understand the overall flow without getting lost in the details on how a post is validated or saved. If you want to know how that works, you can dig deeper. This is also called separation of concern and does not only make the code more readable, it also makes it easier to test.

Separation of Concern

Separation of concern refers to the idea of dividing a program into distinct parts, each of which addresses a specific concern or responsibility. This can make code easier to test because it allows you to focus on testing specific parts of the code independently, rather than having to test the entire program at once.

For example, if you have a program that performs several different tasks, you can divide the code into separate functions or modules, each of which is responsible for a specific task. This allows you to write individual tests for each function or module, and to verify that they are working correctly on their own. This can make it easier to find and fix bugs, because you can isolate the problem to a specific part of the code, rather than having to search through the entire program to find the source of the error.

Additionally, separation of concern can make it easier to test your code because it can help to ensure that your code is modular and reusable. This means that you can use the same functions or modules in multiple parts of your program, and you only need to test them once. This can save time and effort, and it can also help to reduce the amount of code that you need to write and maintain.

Line of Sight

Line of sight is a software engineering principle that suggests that the flow of control through a program should be easy to follow and understand. This means that the structure of the code should be clear and logical, and that the relationships between different parts of the code should be easy to see.

One way to achieve line of sight in your code is to use indentation and white space to visually separate different parts of the code and to show the hierarchy and structure of the program. You can also use clear and descriptive variable and function names to make it easier to understand what each part of the code is doing.

One important idea with the line of sight is to prevent nesting as much as possible.

What is nesting and why do we want to prevent it?

Nesting refers to the practice of placing one control structure inside another, such as putting an if statement inside of a for loop. While this can sometimes be necessary to achieve the desired behavior, excessive nesting can make the code difficult to read and understand.

One reason to avoid excessive nesting is that it can make the code hard to follow. When you have many levels of nested control structures, it can be difficult to keep track of where you are in the code, and to understand how different parts of the code are related to each other. This can make it hard to find and fix bugs, and it can also make it difficult to modify or extend the code in the future.

Another reason to avoid excessive nesting is that it can make the code less modular and reusable. When you have many levels of nested control structures, it can be difficult to extract a specific part of the code and use it in a different part of the program. This can make it harder to write clean and concise code, and it can also make it harder to maintain and update the code over time.

Overall, while some nesting may be necessary in some cases, it is generally best to avoid excessive nesting in your code in order to make it easier to read and understand.

Here is a example of the above code writting with excessive nesting.

func CreatePost(w http.ResponseWriter, r *http.Request) {
    post, err := getPostFromRequest(r)
    if err == nil {
        err = validatePost(post)
        if err == nil {
            err = savePost(post)
            if err == nil {
                http.WriteHeader(201)
            } else {
                handleError("error saving post", err)
            }
        } else {
            handleError("error validating post", err)
        }
    } else {
        handleError("error getting post", err)
    }
}

How do we prevent nesting?

The golden rule is to align the happy path to the left; you should quickly be able to scan down one column to see the expected execution flow. If everything is fine, continue down, if there is an error, go right, handle it and return.

Use early returns
Extract functions to keep bodies small and readable
Avoid else returns; consider flipping the if statement
Put the happy return statement as the very last line

To prevent excessive nesting in your code, you can follow a few best practices. One way to avoid nesting is to use early returns, which means that you return from a function or method as soon as you have completed the necessary work. This can help to reduce the amount of nesting in your code, because it allows you to exit a function or method without having to place the rest of the code inside of an if statement or other control structure.

Another way to avoid nesting is to use helper functions or methods to break up complex code into smaller, more manageable pieces. This can make it easier to write clean and concise code, and it can also make it easier to test and debug your code. By dividing your code into smaller, focused functions or methods, you can reduce the amount of nesting and make your code easier to read and understand.

Interviewing as a Team Lead or Engineering Manager

Jaap Groeneveld — Tue, 06 Dec 2022 20:00:00 +0000

I have an interview for a software engineering team lead position coming up. What questions should I be prepared for?

In a team lead position, you'll likely be asked questions that evaluate your leadership and management skills, as well as your technical expertise. Some common questions you might be asked include:

Can you tell me about a time when you led a team or project, and what you did to ensure its success?
How do you motivate and manage team members to achieve their goals?
How do you handle conflicts within a team and ensure that everyone is working together effectively?
Can you provide an example of a difficult technical problem you faced and how you solved it?
How do you stay up-to-date with the latest technology and trends in software engineering?
What is your approach to project management and planning?
How do you communicate with other teams and stakeholders to ensure that everyone is on the same page and working towards the same goals?
Can you give an example of a time when you had to make a difficult decision as a team lead, and how you went about making it?
How do you balance the needs of the team with the needs of the business or organization?
Can you describe a time when you had to deal with a underperforming team member, and how you handled the situation?

Team Lead interviews often rely on behavioral interviews questions designed to elicit specific examples of the candidate's past behavior in order to determine their suitability for the job. The idea behind this type of interview is that a person's past behavior is the best predictor of their future behavior, so asking the candidate to provide specific examples of how they have handled certain situations in the past can provide insight into how they might handle similar situations in the future. Behavioral interviews are often used to assess a candidate's skills, knowledge, and experience, and can help the interviewer determine whether the candidate is a good fit for the job.

The STAR model is a technique that is often used in behavioral interviews to help structure answers to questions. It stands for Situation, Task, Action, and Result.
When answering a question using the STAR model, you would first describe the specific situation or task that you were faced with, then describe the actions that you took to address the situation, and finally, describe the results of those actions. This technique can help you provide a clear and concise answer to the interviewer, and can also help you demonstrate how your skills and experiences are relevant to the job you are applying for.

Answers to some common questions

How do I motivate and manage team members to achieve their goals?

Ultimately, the key to motivating and managing team members is to create an environment where they can do their best work, and where they feel valued and supported. By focusing on clear goals, open communication, and ongoing support, you can help your team members achieve their goals and succeed.

To motivate and manage team members to achieve their goals, there are a few key things you can do:

Set clear and specific goals for the team and for individual team members. This will help everyone understand what they need to do, and why it's important.
Communicate regularly with team members to check in on their progress and provide support and guidance.
Provide opportunities for professional development and training, so that team members can grow and improve their skills.
Recognize and reward good performance, and provide constructive feedback to help team members improve.
Create a positive and collaborative work environment, where team members feel valued and supported.
Be a good role model and lead by example. Show team members what it looks like to work hard, be focused, and deliver high-quality work.
Be open to feedback and suggestions from team members, and be willing to make changes based on their input.

How do I handle conflicts within a team and ensure that everyone is working together effectively?

Conflict within a team is inevitable, but it can be managed and resolved in a way that ensures that everyone is working together effectively. By addressing conflicts openly and honestly, and working together to find solutions, you can help ensure that your team is able to work together effectively and achieve its goals.

Here are some steps you can take to handle conflicts within a team:

Address conflicts as soon as they arise. Don't ignore them or hope that they will go away on their own.
Be open and transparent about the conflict. Encourage team members to express their thoughts and feelings, and listen to what they have to say.
Identify the root cause of the conflict. Is it a misunderstanding, a personality clash, or a disagreement about how to approach a problem? Understanding the underlying issue can help you find a solution.
Find a resolution that works for everyone. This may involve compromising, negotiating, or finding a creative solution that meets the needs of all parties involved.
Follow up to ensure that the conflict is resolved and that everyone is moving forward in a positive way.

Can you provide an example of a difficult technical problem you faced and how you solved it?

Use the STAR model to describe a concrete situation and make sure to highlight your part in the situation.

How do you stay up-to-date with the latest technology and trends in software engineering?

You need to answer this with your personal situation. Here are a few different ways to stay up-to-date with the latest technology and trends in software engineering:

Attend conferences and workshops focused on software engineering and related topics. These can be great opportunities to learn about new technologies and best practices, and to network with other professionals in the field.
Read blogs, newsletters, and other online resources that cover the latest developments in software engineering. This can help you stay on top of new technologies and trends, and can provide valuable insights and ideas.
Join online communities and forums related to software engineering. These can be great places to connect with other professionals, share knowledge and experiences, and learn from each other.
Follow thought leaders and influencers in the field of software engineering on social media. This can help you stay informed about the latest developments and ideas, and can also help you connect with other professionals in the field.
Experiment with new technologies and tools on your own. This can be a great way to gain hands-on experience and stay up-to-date with the latest developments in software engineering.

What is your approach to project management and planning?

In general, effective project management and planning involves a few key steps:

Define the scope of the project, including the goals and objectives, the deliverables, and the timeline. This will help you and your team understand what needs to be done, and when it needs to be done.
Develop a detailed project plan, including a schedule, a budget, and a list of tasks and milestones. This will help you organize your work and track your progress.
Identify the resources you'll need, including people, equipment, and materials. This will help you make sure you have everything you need to complete the project.
Communicate with your team, stakeholders, and other relevant parties throughout the project. This will help everyone stay informed and on the same page.
Monitor and track your progress, and make adjustments as needed. This will help you stay on schedule and on budget.
Review and evaluate the project once it's complete, and use the lessons learned to improve future projects.

How do you communicate with other teams and stakeholders to ensure that everyone is on the same page and working towards the same goals?

Effective communication is key to ensuring that everyone is on the same page and working towards the same goals. Here are some tips for communicating with other teams and stakeholders:

Be clear and concise in your communication. Avoid using technical jargon or complex language that others might not understand.
Use a variety of communication methods, such as face-to-face meetings, email, and online collaboration tools, to reach different people in different ways.
Listen to what others have to say, and be open to their feedback and suggestions. This will help you understand their perspectives and build trust.
Provide regular updates on the progress of the project, and highlight any potential issues or challenges that need to be addressed.
Be transparent and honest in your communication. If there are problems or delays, let others know as soon as possible, and provide clear explanations for what's happening and how you plan to address it.

Can you give an example of a time when you had to make a difficult decision as a team lead, and how you went about making it?

In general, when faced with a difficult decision as a team lead, it's important to consider all of the available information and options, and to weigh the potential pros and cons of each. Here are some steps you can take to help you make a difficult decision:

Define the problem or issue that needs to be addressed. This will help you clarify what needs to be decided, and why it's important.
Gather as much information as possible. This may involve talking to team members, stakeholders, and other relevant parties, as well as researching the issue.
Consider all of the options and alternatives. Try to be as creative and open-minded as possible, and think about the potential implications of each option.
Consult with others, as appropriate. This may involve talking to other team leads, managers, or experts in the field.
Make a decision and take action. Once you've considered all of the information and options, make a decision and take action. Be prepared to explain your decision to others and to defend it if necessary.

Use the STAR model to describe a concrete situation and make sure to highlight your part in the situation.

How do you balance the needs of the team with the needs of the business or organization?

As a team lead, it's important to balance the needs of the team with the needs of the business or organization. This can be challenging, but there are a few key things you can do to help you strike the right balance:

Understand the goals and objectives of the business or organization, and make sure that the team's goals and objectives align with them.
Communicate regularly with stakeholders and other relevant parties, and make sure that everyone is on the same page and working towards the same goals.
Prioritize tasks and projects based on their importance to the business or organization, and allocate resources accordingly.
Be flexible and adaptable. Be willing to make changes and adjustments as needed to meet the changing needs of the business or organization.
Involve the team in decision-making and problem-solving. This will help ensure that the team's needs are taken into account, and that everyone feels invested in the success of the business or organization.

Can you describe a time when you had to deal with a underperforming team member, and how you handled the situation?

In general, when dealing with an underperforming team member, it's important to approach the situation with empathy, respect, and professionalism. Here are some steps you can take to handle the situation:

Identify the underlying issue. Is the team member struggling with a specific task or skill, or are there other factors that are impacting their performance?
Talk to the team member and discuss the issue openly and honestly. Be specific about their performance, and provide concrete examples of what they're doing well and what they need to improve.
Listen to the team member's perspective, and try to understand their challenges and concerns. Be empathetic and supportive, and avoid being critical or judgmental.
Develop a plan to help the team member improve their performance. This may involve providing additional training, support, or resources, or setting specific goals and objectives.
Follow up regularly to check on the team member's progress, and provide ongoing support and guidance.
Evaluate the team member's performance over time, and be prepared to make changes to the plan if necessary.

Use the STAR model to describe a concrete situation and make sure to highlight your part in the situation.

About documentation

Jaap Groeneveld — Thu, 10 Nov 2022 11:41:40 +0000

In this Article I am talking about comments but also check out my article on comments.

Thoughts on documentation

Developers hate writing documentation. And I understand it. Writing it is very hard. Reading bad documentation is never done because nobody reads it anyway. When you are working on a project and you are knee deep in the project, you know everything anyway, why document?

I am a fan of good documentation. But to understand, what is good documentation, lets talk about the different types of documentation.

Types of documentation

Inline Documentation: Inline Comments

Comments within the code are also one type of documentation that is often overlooked. You can read my article about them here.
They are used directly within the code to clarify things for the reader.

In a nutshell:

They are very close to the code
They are very detail oriented.
You won't read them in advance when starting a project.

Inline Documentation: Doc Comments

These are used within the code to document higher level components like functions, classes, modules. Quite often, they rely on a standard and can be used to generate documentation in a dedicated place.

Close to the code
Describe the components within a codebase on a higher level.
You might read them one by one when using or working on a component or skip through the high level components like modules when starting on a project.

API Documentation

This one is very obvious. Documenting the API a library or service provides makes total sense. It's the best way to communicate how to use it to a consumer.

Often generated within the code by annotating web handlers / controllers.
They only document the outer interface of a service or library
You might read them when using something new or to look up details.

Pitfalls: Often the API documentation does not contain a complete example on how to use it and how to combine multiple requests. This is very helpful.

Readme Documentation

They usually describe one service in a repository. If you have multiple services in one repository, there might be multiple readmes, one per service. Most often they describe what the service is supposed to be doing and how you get started running and developing it and also contain pointers to further documentation.

They are close to the code
The describe the purpose of the service and how to get started
You will read them when you get started developing.

Pitfalls: If the documentation in the readme is to detailed, it will get out of sync with reality very quickly. Therefor it is best practice to only document what is stable and also have every new developer update the readme when they onboard.

Application Architecture Documentation

This is where documentation starts to get more visual. If the application is not trivial, it is very helpful to document the application architecture. Usually this emphasizes the layers, bigger modules and direct external dependencies like caches, data store, queues, other API's etc.

This documentation is not very close to the code anymore as it is high level and therefore can not be autogenerated in a meaningful way.
It has to be maintained by hand when something changes.
Pitfalls: If this is to detailed it will be wrong very soon. If a new dependency is added, often it is forgotten to add it in this documentation. It is very useful to use this document when onboarding new developers and periodically have a team mob-session to update it. This refreshes memory and keeps it more or less up to date.

System Architecture documentation

Depending on the size of the system this part documents the services communicating with each other within the org, the department or the team. Sometimes there will be multiple System Architecture Documentations to limit the scope you have to understand. Sometimes it makes sense to add more documentation for specific parts of the architecture.

Very far away from the code, focussing on very high level and the communication between systems.
Can be augmented with automatic services that analyze services talking to each other and draw diagrams like Backstage.
Pitfalls: If this is to detailed it will be wrong very soon. If new services are added, often it is forgotten to add it in this documentation. It is very useful to use this document when onboarding new developers and update it accordingly.

Infrastructure & Deployment documentation

It is very helpful to quickly understand where and how a service is deployed, when are which tests running and have a visual representation of that.

Where is the service deployed? What infrastructure is involved.
What is the process of deployement, how does the pipeline look like? What checks are running etc.
Sometimes the visual pipeline documentation of Tools like Github Actions, Jenkins or Concourse can be good enough or helpful and should be at least linked.

Process Documentation / Runtime Views

Every application and every system contains processes. Some more complicated than others, some more important than others. What happens when a user registers? Which services are hit? Are workers involved?

Pitfalls: Do not fall into the trap and try to be very precise or try to document each process that happens. Practice MVD - Minimum Viable Documentation. Focus on the most important and most used/changed processes.

Decision Documentation

Documentation and also Code do not document the history of an application. It is also not possible to economically document all bits and bobs of a system. Therefore it make sense to focus on decision documentation that is easy to search.

Benefits:

It is obvious that it was just a moment in time, it might be out of date
It documents the history, the reason for the decison and also why alternatives have not been chosen.

What kind of decisions should be documented?

Architecture decisions
Tech decisions (Language, libraries, frameworks..)
Process Decisions.

Pitfalls: In a software projects are a lot of decisions made on a daily basis. It is not easy to see which decision is a documentation worthy decision. in the beginning, decisions should be discussed with the team and documented to get a feeling what are bigger decisions.

One example for Decision Documentation are ADR - Architecture Decision Records.
Here is an example.

MVD - Minimum Viable Documentation

Have you heard about MVD - Minimum Viable Documentation? No? Don't worry, it is a term that I just made up.

Just like with MVP - Minimum Viable Product, the idea behind MVD is to be very mindful about what amount of documentaton actually solves the problem that it aims to solve. Documentation aims to help you, when onboarding into a new project or during the work on a new project to understand specific parts to you do not have in mind. Too much documentation will just clutter your mind. Therefore documentation should be optimized for two use cases:

Onboarding: Optimize for quick and high level understandability of the system and application.
Look up: Optimize for searchability and correctness. Accept incompleteness in the documentation but make it obvious.

For Onboarding, good documentation should focus on high level and visual documentation. For Look Up, documentation should additionally focus on decision records and relevant process documentation.

Documentation should be as lightweight as possible - do not use big templates that are filled with irrelevant filler material. It should be easy to read, search and edit.

Guidelines for good documentation

Be as transparent as possible on what documentation exists and make it hard to miss.
Kep it as close to the code as possible (what you see is what you know exists)
Only document things that are very stable and do not change too much. The more something changes the more time you need to spend on keeping documentation correct.
The more documentation exists, the less people will read it. Practice MVD - Minimum Viable Documentation.
Embrace fuzzy Documentation for things that do change.
Document what somebody withour prior knowledge would need to understand the system quickly, without going into details.
Use onboarding of new developers as a documentation update tool. Use the High level Application Architecture, System Architecture and Deployment Documentation for onboarding and update before and during onboarding.
Do periodic team mob-sessions to refresh the memory on the high-level architecture and also to update it if it changed.
Try to standardize the way documentation works across the organization.

Documentation frameworks

I prefer arc42 - it not only helps you with architecture and process diagrams but also is a great tool to use when starting a new project. It also covers risks, assumptions, stakeholders, decisions made etc.

Other helpful tools and services

Backstage - Service catalog and documentation. List API providers & Consumers & API's, list documentation, etc.
Swagger API Documentation Generation

About code comments

Jaap Groeneveld — Thu, 27 Oct 2022 11:11:32 +0000

Documentation and comments serve a similar purpose. It is about communicating to other developers now, to other developers in the future and also to yourself in the future.

When I am talking about comments, I am not talking about comments used to document an API, functions, classes or modules. These are not really comments but rather inline documentation.

In this Article I am talking about comments but also check out my article on documentation.

Comments

Code comments have a really bad reputation and there is a good reason for it.
Comments, just like documentation, lie really quickly. Most often because they get outdated and are not updated.

Often, they only state the obvious and this just clutters the code.

#loop through the values to be checked
while i <= maxNum:
    [...]

For this reason, a lot of people have outlawed comments and will not comment their code at all. I understand this but still comments have a really good use and I like to make a case for using them sparingly.

Do not use comments if not necessary

Sometimes better names make comments redundant and even improve the code because using the name always carries the meaning.

let price = 0; // price in cents

let priceInCents = 0;

Often, comments are used to structure code. For structure we have better tools that never lie.

function processInput(input: string) {  
    // validate input  
    if (input.length < 10) {  
        throw "input too short"  
    }  
    if (input.length > 100) {  
        throw "input too long"  
    }  

    // remove colors  
    let result  = input.replace('red', '').replace('green', '').replace('blue', '');  

    // format result  
    result = result.trim().toUpperCase()  

    return result  
}

If we just take the comments and make them into functions, the code is even more clear. You don't even have to look at the implementation details if you do not care at this point in time. It can also be used to contain ugly and unclear implementations (sometimes it is just not avoidable).

function processInput(input: string) {  
    validateInput(input);  

    let result = removeColors(input);  

    return formatResult(input)  
}

function validateInput(input: string) {  
    if (input.length < 10) {  
        throw "input too short"  
    }  
    if (input.length > 100) {  
        throw "input too long"  
    }
}

function removeColors(input: string) {  
    return input.replace('red', '').replace('green', '').replace('blue', '');  
}  

function formatResult(result: string) {  
    return result.trim().toUpperCase();  
}

The above code very clearly shows the 3 responsibilities of the processInput function. validate, do and return formatted. I do not want to make the point that you always have to structure your functions this way. Often, having the validation inside the "mother" function is a good idea to show exactly what is going on. But when you feel the urge to write a comment to structure code, think about using a function for structure.

Use comments to explain the why, the odd and the consequences

What we, more often than not, can not illustrate with code structure is the why, the odd and the consequences of something. Everytime something would surprise you in a year or a new developer, this is a very good reason to introduce a comment.

the why

// we are moving between API's so we are calling new and old apis.
oldUsersApi.updateData(userData);
newUserApi.updateData(userData);

the odd

// this is odd but the 3rd party client api enforces us to disconnect before we can connect
client.disconnect()
client.connect()

the consequences

user.delete() // deleting the user will cascade deletion of all their data

My guidelines for commenting code well

Only comment the why and only if not obvious
Always try to explain yourself in code first.
Use as explanation of intent.
Use as clarification of code.
Use as warning of consequences.

This article is extracted from a series of talks I gave.

Jaap Groeneveld - Software Engineering Principles

A talk about software engineering principles & clean code

slideshare.net

About naming things

Jaap Groeneveld — Sat, 01 Oct 2022 20:42:56 +0000

There is this one quote that I hear often repeated.

There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton

But is it really true? Yes, finding the perfect name for something is difficult. But let us take a step back and think about why we are naming things and why we are looking for good names.

Names are for humans

The computer usually does not care about how things are named. Therefore the name is just for the human reading and modifying the code.
Therefore, when choosing a name for something we have the goal to choose descriptive and unambiguous names.

How do we do this?

Make names human

Use pronounceable names.
Use searchable names.
Avoid encodings. Don't append prefixes or type information.

Having human-readable names make them easier to remember, easier to think about and also easier to discuss with a college.

One word per concept

Make meaningful distinction.
Having a clear terminology is key to not get confused.

Its more important to be consistent then to be perfect. The only thing that matters is that someone joining the project can learn what the concepts behind things are and that they can infer from one thing to another what it might be about.

I do not care if the things executing your business logic are called Services, Interactors, Actions, Commands or Use Cases. Just stay with one name. The difference do not matter in the end. It just matters that everyone understands where to put and search for business logic.

Another example. How to make clear that getting some data involves more than just returning it from memory? Choose a name like for example fetch. From now on, every method that gets data from a another API or the database is called fetchX and everything that returns data from memory is called getX. You have something that is being calculated based on input data? call it calculateX
Are these names 100% clear for everyone that is not working on this project? No, they are not. Are these perfect names for the concepts? No, they are not but as long as you keep the naming and meaning consistent, everybody working on the projet will understand and it wil help out.

Introducing redundant variables

Often, introducing variables for returned data is technically not needed because you call or return the result directly.

func doSomething() {
    defer StartTimer()()
    // [...]
}

Here, we call the function StartTimer. It returns a function that does something and we call it on the exit of the function with defer. What does the returned function do? We could guess. Maybe we know the StartTimer function already so we know. We could go into StartTimer and have a look or read the documentation. The Point is, you need to know or look it up.

If you introduce a name for the returned function:

func doSomething() {
    stopTimer := StartTimer()
    defer stopTimer()
    // [...]
}

This problem goes away. Everybody understands the resulting function instantly. No chance for confusion. Yes, we added 1 extra line, but that does not hurt anyone and we introduced a lot more clarity to our code. One less thing you have to know or look up.

Using type aliases

Some languages allow introducing type aliases (for example Kotlin, Typescript, Go), giving a existing type an additional name.

We can leverage this to make our functions more clear. Often even leverage the type system to catch errors:

func getUserData(companyID string, userID string) {}

// vs  

type CompanyID string
type UserID string
func getUserData(companyID CompanyID, userID UserID) {}

The benefits are:

Impossible to "accidentally" assign something else
Easy to find where User IDs are used (searchability through static analysis)
Prevents positional problems in functions, it is not possible to accidentally mix up companyIDand userID.
Depending on the language, they allow adding functions to the type, for example for validation or normalization.

This article is extracted from a series of talks I gave.

Jaap Groeneveld - Software Engineering Principles

A talk about software engineering principles & clean code

slideshare.net

Programming or Software Engineering?

Jaap Groeneveld — Sat, 01 Oct 2022 10:33:00 +0000

Software engineering is what happens to programming when you add time and other programmers. - Russ Cox

Coding something from scratch is (usually) easy. Getting things to work is (usually) easy. Things get complicated when you think about the past, the future and other people.

Software Engineering deals with the challenges of

Communication between people
Communication over time
Changing requirements
Taming complexity

Clean code is more about humans than its about tech.

Why invest in clean code?

Minimizes time spend on reading and understanding
Easier to grow in the future
Less hunting for bugs
Easier onboarding.
=> Messy code is Technical debt. You will pay for it down the road.

Messy code has the uncanny ability to slow down any developer and make his work much harder

Bad Code smells

A code smell is any characteristic in the source code of a program that possibly indicates a deeper problem.

Here is a list of characteristics of a codebase that might lead you to think there is time to change:

Rigidity. The software is difficult to change. A small change causes a cascade of subsequent changes.
Fragility. The software breaks in many places due to a single change.
Immobility. You cannot reuse parts of the code in other projects because of involved risks and high effort.
Opacity. The code is hard to understand.
Needless Complexity.
Needless Repetition.

Principles to prevent bad code

Keep it simple, stupid (KISS) and Clear is better than clever.

Writing clever code feels good. You achieved something. But more often than not, clever code increases the opacity of the code. If another developer has to stop at a piece of code and has to understand it because it is not communicating clearly what is going on, it either is a very complex matter or the code is more complex than necessary.

Lets take this snippet:

['red', 'green', 'blue'].reduce(
  (myValue, color: string) => myValue.replace(color, ''),
  myValue,
)

What is it doing? It reduces the array containing colors by applying a function that is calling replace on myValue for the color it is iterating over and replaces the color with an empty string. So, after understanding that, what it is doing is removing all occurances of the colors in the array from myValue.

The following snippet is simpler and understood instantly. Yes, it is repeating code. So what? In this case, it does not matter.

myValue.replace('red', '').replace('green', '').replace('blue', '');

You aren’t gonna need it (YAGNI)

Do not build things that you think you might need in the future. It overcomplicates the code and experience shows that often things will not be build in the future. Requirements and priorities change. The understanding of the problem will change.

Premature Optimization

The same is true for optimization. Of course it is fine to optimizate code when it does not cost anything. If you know your code is faster if you use a Hashmap and access directly instead of using a List, do it. Writing it costs nothing. But starting out writing complicated solutions to optimize a request by 100ms is not the right way. Only optimize what is actually a problem and only as soon as you know it is a problem.

Don't repeat yourself (DRY), single source of truth.

This is often misunderstood as not having the same lines of code in the program multiple times. Do not repeat the same characters. But this is actually not the right way to interpret the DRY principle. Don't confuse syntactic and semantic dry-ness.

DRY is about not repeating the same business logic in multiple places. There should be only one place in the program that describes how a business process is happening. This is about avoiding magic numbers, avoiding having the calculation of number of posts a user has in multiple places. The reason behind this is that when you have to change the program, you will probably forget to change all occurances of the logic and the number of posts will diverge.

If the code is simple it might be better for clarity to actually copy it. Do not overstructure things that don't matter.

Practice consistency.

Keep your code consistent. The way you code, the way things are structured helps you and other developers. Onboarding will be easier. Humans are great at extracting systematic structure but are really bad at remembering all the special cases.

Keep the tests clean.

If you let the tests rot, then your code will rot too.
Tests are code too. You will have to maintain them just like your production code, so treat them like production code. Introduce abstractions, stay DRY, have a high level of quality. Tests are often the best documentation you have, so clarity is extremly important. Make it very obvious what you are actually testing and hide the details. You will thank yourself soon. In my experience, most frustrations during development come from a bad test base.

Prepare for change

All software projects I worked on had in common that they never have been finished. There is always change involved. Therefor it is very important to always ask how will this code behave if X changes (but keep YAGNI in mind). If you keep the code somewhat flexible without over-engineering, you will win the game of software development. This is more art then science though and comes with experience. Also, you will be wrong and that is fine!

This article is extracted from a series of talks I gave.

Jaap Groeneveld - Software Engineering Principles

A talk about software engineering principles & clean code

slideshare.net

How to use minitest/mock to mock any instance

Jaap Groeneveld — Wed, 02 Mar 2022 14:27:31 +0000

stub_any_instance is a very helpful utility in common mocking frameworks. minitest/mock does not provide this but its very simple to provide a similar effect.

Mocking/Stubbing arbitrary instances of classes is very helpful in testing. In ruby it's easy to do this even without using some sort of dependency injection.

Let's do it!

Lets assume we have a Fetcher class that needs an instance of ApiClient

class Fetcher
  def fetch_data
    ApiClient.new.get("https://service.com/api/data")
  end
end

We start by creating a mock and setting the relevant expectations like always.

mock_api_client = Minitest::Mock.new
mock_api_client.expect(:get, {data: "return-data"}, ["https://service.com/api/data"])

Now we need to make this mock the instance that some class would instantiate by stubbing the class method .new

ApiClient.stub(:new, mock_api_client) do
  result = Fetcher.new.fetch_data
  assert_equal {data: "return-data"}, result
end

The mock/stub combination would be active during the duration of the block.

How to run rails tests in Visual Studio Code

Jaap Groeneveld — Wed, 02 Mar 2022 14:27:22 +0000

Visual Studio is great. It's very slim and quite fast, at least when updated to the M1 version.
The testing support for rails seems to be quite outdated. There is a plugin Ruby Test Explorer but it looks like it needs updating for using the new rails test runner rails test. In the end, it's quite simple to actually run tests on your own and have 100% control over it. This will also allow you to rerun the last test, which is a critical productivity feature.

What you will get

With the method I show here, you can customise your test running as you wish. In the end, you will have the following shortcuts:

run all tests: ctrl+t ctrl+a (bin/rails test)
run all tests in current file: ctrl+t ctrl+f (bin/rails test {file})
run test under cursor: ctrl+t ctrl+l (bin/rails test {file:line})
rerun last test: ctrl+t ctrl+t

The method

Create a custom script

To gain the ability to rerun tests, we have to somehow store what we already ran. I am sure, there is an easier way within VSCode, but this method works for sure:

Create a file bin/editor-test
touch bin/editor-test && chmod +x bin/editor-test

Add the following script:

#!/usr/bin/env bash

# used to run tests with editor and saving the last run so we can repeat.
# for each test run it will save the last args in "tmp/editor-test-last-args".
# usage
# run file:
#   bin/editor-test some_file.rb:12
# run all:
#   bin/editor-test
# run last:
#   bin/editor-test run-last

set -e

if [ "$1" = "run-last" ]; then
  if [[ -f "tmp/editor-test-last-args" ]]; then
    bin/rails test "$(<tmp/editor-test-last-args)"
  else
    echo "No last test saved (file tmp/editor-test-last-args does not exist)"
  fi
else
  echo $@ > tmp/editor-test-last-args
  bin/rails test $@
fi

This provides you will the possibility to rerun tests from the console by storing the last test args in tmp/editor-test-last-args. Now, lets hook up vscode.

Setup custom VSCode Tasks

This is a per-project task file, so you could customize this per project.

Run the vscode command (cmd+shift p) "Tasks: Configure Task" and configure the following tasks:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "repeat last test",
            "type": "shell",
            "command": "bin/editor-test run-last",
            "problemMatcher": [],
            "presentation": {
                "clear": true
            },
        },
        {
            "label": "launch tests for project",
            "type": "shell",
            "command": "bin/editor-test",
            "problemMatcher": [],
            "presentation": {
                "clear": true
            },
        },
        {
            "label": "launch tests for current file",
            "type": "shell",
            "command": "bin/editor-test ${file}",
            "problemMatcher": [],
            "presentation": {
                "clear": true
            },
        },
        {
            "label": "launch test for current line",
            "type": "shell",
            "command": "bin/editor-test ${file}:${lineNumber}",
            "problemMatcher": [],
            "presentation": {
                "clear": true
            },
        }
    ]
}

Setup keybindings

To have perfect productivity, lets create some keybindings:

{
    "key": "ctrl+t ctrl+a",
    "command": "workbench.action.tasks.runTask",
    "args": "launch tests for project"
},
{
    "key": "ctrl+t ctrl+f",
    "command": "workbench.action.tasks.runTask",
    "args": "launch tests for current file"
},
{
    "key": "ctrl+t ctrl+l",
    "command": "workbench.action.tasks.runTask",
    "args": "launch test for current line"
},
{
    "key": "ctrl+t ctrl+t",
    "command": "workbench.action.tasks.runTask",
    "args": "repeat last test"
},

Conclusion

There you have it. Super simple, easy to customise test running. This approach will work with every technology, not only rails and no need for plugins.