DEV Community: Mark Adel

A Philosophy of Software Design Summary

Mark Adel — Sat, 19 Jul 2025 12:18:15 +0000

I just finished reading A Philosophy of Software Design by John Ousterhout.

It was an easy and insightful read. The book reinforced many concepts I was already familiar with and introduced a few new ideas that I was able to immediately apply at work to reduce complexity and write more maintainable code.

I carefully prepared this summary of the book using AI, ensuring that it captures all the main ideas.

In my opinion, this summary doesn’t substitute for reading the book itself. I still highly recommend reading it in full.

Chapter 1, 2: Introduction and The Nature of Complexity

As programs evolve and gain features, complexity accumulates, making them harder to understand and modify, which slows down development and leads to bugs.
Symptoms of Complexity:
- Change Amplification: A simple change requires modifications in many different places.
- Cognitive Load: Developers need to know a lot of information to complete a task, increasing learning time and bug risk.
- Unknown Unknowns: It's not clear what code to modify or what information is needed for a task, leading to bugs discovered only after changes are made.
Causes of Complexity:
- Dependencies: When a piece of code cannot be understood or modified in isolation, requiring consideration or modification of other code. Dependencies should be minimized, and made obvious when introduced.
- Obscurity: Important information is not obvious, often due to vague names, missing documentation, or inconsistencies.
Complexity is Incremental: Complexity accumulates in small chunks (hundreds or thousands of small dependencies and obscurities), making it hard to control and eliminate unless a "zero tolerance" philosophy is adopted.

Chapter 3: Working Code Isn't Enough

Tactical Programming: Focuses on getting features working as quickly as possible, often leading to short-sighted decisions and the rapid accumulation of small complexities.
Strategic Programming: Requires an investment mindset where time is spent on improving system design and fixing problems, even if it slows down short-term progress. The primary goal is a great design that also works.
Benefits of Strategic Programming: Proactive investments (e.g., finding simple designs, writing good documentation) and reactive investments (fixing design problems when discovered) lead to continuous improvements and long-term development speed.
Investment Amount: Spending about 10-20% of total development time on design investments, which quickly pays for itself and leads to faster development in the long run.
Consistency is Crucial: It's important to be consistent in applying the strategic approach and to see investment as a continuous, not delayed, task.

Chapter 4: Modules Should Be Deep

Interface vs. Implementation: Each module has an interface (what a developer needs to know to use it, describing what it does) and an implementation (how it fulfills its promises). Developers should only need to understand the interface of other modules.
Formal and Informal Interface Elements: Formal parts are explicitly specified in code (e.g., method signatures, public variables) and can be checked by the language. Informal parts (e.g., high-level behavior, usage constraints) are described in comments.
Abstractions: A simplified view of an entity that omits unimportant details, making complex things easier to think about. A module's interface is its abstraction.
Deep Modules: Modules that provide powerful functionality but have simple interfaces. Their interfaces are much simpler than their implementations, hiding significant complexity and allowing internal changes without affecting other modules. Examples include:
- Unix I/O. A deep interface with five simple system calls (open, read, write, lseek, close) that hide hundreds of thousands of lines of complex implementation code.
- Garbage Collector. A module with no interface that works invisibly to reclaim memory, effectively shrinking the overall system interface by eliminating the need for explicit object freeing.
Shallow Modules: Modules whose interfaces are relatively complex compared to the functionality they provide, hiding little complexity. They offer little leverage against complexity. Examples enclude:
- A class that only contains one static method, formatToUpperCase(String input). This class is shallow because it introduces the overhead of a new class and method call for functionality that could easily be a one-liner
- A LoggingService class that simply calls System.out.println() for every method, without adding any actual logging logic.
Classitis: A syndrome where classes are designed to be excessively small, based on the mistaken view that "more classes are better." This results in many shallow classes, increasing overall system complexity due to numerous interfaces and verbose code.
- Example: Java I/O: The Java class library often exhibits classitis, requiring developers to compose multiple objects (e.g., FileInputStream, BufferedInputStream, ObjectInputStream) for a simple task like reading serialized objects, making it verbose and error-prone.
Conclusion: Users of a module need only understand the abstraction provided by its interface. The most important issue in designing classes and other modules is to make them deep, so that they have simple interfaces for the common use cases, yet still provide significant functionality.

Chapter 5: Information Hiding (and Leakage)

Information Hiding: A crucial technique for creating deep modules, where each module encapsulates design decisions (knowledge) within its implementation, preventing it from appearing in its interface and thus being invisible to other modules.
Benefits of Information Hiding:
- Simplifies the module's interface, reducing cognitive load for users.
- Makes system more robust and scalable by reducing dependencies.
Information Leakage: Occurs when a design decision is reflected in multiple modules, creating dependencies.
Addressing Information Leakage: Reorganize classes so that knowledge affects only a single class, possibly by merging closely tied classes or creating a new class that encapsulates the leaked information with a simple, abstract interface.
Taking it Too Far: Information hiding is only beneficial if the information is genuinely not needed outside the module. If critical information (e.g., performance-affecting configuration parameters) is hidden, it can impede proper use and tuning. The goal is to minimize needed external information.

Chapter 6: General-Purpose Modules are Deeper

Specialization Leads to Complexity: Over-specialization is a significant cause of complexity in software.
Generality Simplifies Code: More general-purpose code is simpler, cleaner, and easier to understand, applying at all levels of software design (classes, methods, eliminating special cases in code).
Somewhat General-Purpose Classes: The "sweet spot" is to implement new modules in a "somewhat general-purpose" fashion. Functionality should meet current needs, but the interface should be general enough for multiple potential uses, without being tied specifically to current needs.
Generality and Information Hiding: General-purpose APIs lead to better information hiding by separating concerns and encapsulating specific details where they belong. False abstractions hide information that users actually need, creating obscurity.
Questions for General-Purpose Design:
- What is the simplest interface that covers all current needs? (Aim for fewer, more general-purpose methods but without sacrificing ease of use)
- In how many situations will this method be used? (If a method is designed for one particular use, it may be too special-purpose. See if you can replace several special-purpose methods with a single general-purpose method)
- Is this API easy to use for my current needs? (If too much additional code is needed, the API may be too general or provide the wrong functionality)
Push Specialization Upwards (and Downwards!): Specialized code should be cleanly separated from general-purpose code by pushing it higher (into application-specific classes, like UI code) or lower (into specific implementations, like device drivers).

Chapter 7: Different Layer, Different Abstraction

Layered System Design: Software systems are typically composed in layers, where higher layers use the facilities of lower layers. In good design, each layer provides a distinct abstraction.
Red Flag: Similar Abstractions in Adjacent Layers: If adjacent layers have similar abstractions, it's a red flag indicating a problem with class decomposition.
Pass-Through Methods: A common manifestation of similar abstractions in adjacent layers. A method does little more than invoke another method with a similar or identical signature.
Problems with Pass-Through Methods: They make classes shallower, increase interface complexity, and create dependencies between classes (if the underlying method's signature changes, the pass-through method must also change). They indicate confusion over class responsibility.
Solutions for Pass-Through Methods:
- Expose the lower-level class directly to callers.
- Redistribute functionality between classes for distinct responsibilities.
- Merge classes if they cannot be cleanly disentangled.
When Interface Duplication is OK: It's acceptable if each method provides useful and distinct functionality, even with similar signatures (e.g., dispatchers that choose one of several methods to invoke, or different implementations of the same interface).
Decorators (Wrappers): A design pattern that extends an object's functionality by wrapping it, often providing a similar API to the underlying object.
Problems with Decorators: They tend to be shallow, introducing boilerplate for little new functionality, and often contain many pass-through methods. Overuse can lead to an explosion of shallow classes.
Alternatives to Decorators:
- Add new functionality directly to the underlying class if general-purpose or naturally related.
- Merge specialized functionality with the use case.
- Merge with an existing decorator to create a deeper one.
- Implement as a standalone class if truly independent.
Interface vs. Implementation Abstraction: The interface of a class should usually differ from its internal implementation. If they are too similar, the class is likely shallow. Good design encapsulates implementation complexity behind a simpler, higher-level interface.
Conclusion: Every design element adds complexity; it must eliminate more complexity than it introduces to be worthwhile.

Chapter 8: Pull Complexity Downwards

Developer Responsibility: Module developers should strive to make life as easy as possible for the users of their module, even if it means extra work for themselves.
Simple Interface over Simple Implementation: It is more important for a module to have a simple interface than a simple implementation.
Avoiding "Punting" Complexity: Developers often "punt" hard problems (e.g., throwing exceptions, exporting configuration parameters) to callers or administrators, which simplifies their own code in the short term but amplifies complexity across the system.
Example: Configuration Parameters: Exporting configuration parameters (e.g., cache size, retry interval) can be a form of moving complexity upwards. Before exporting a configuration parameter, ask yourself: “will users (or higher-level modules) be able to determine a better value than we can determine here?”. Avoid configuration parameters where possible. If necessary, provide reasonable defaults. Each module should aim for a complete solution.
Taking it Too Far: Pulling complexity downward is beneficial if the complexity is closely related to the class's existing functionality, simplifies other parts of the application, and simplifies the class's interface. Overdoing it can lead to information leakage or a single overly complex class.
Conclusion: When developing a module, look for opportunities to take a little bit of extra suffering upon yourself in order to reduce the suffering of your users.

Chapter 9: Better Together Or Better Apart?

Fundamental Question: Should two pieces of functionality be implemented together or separately? The goal is to reduce overall system complexity and improve modularity.
Bring Together if Information is Shared: If multiple methods or classes share knowledge of a particular format or structure, combining them reduces information leakage and simplifies code.
Bring Together to Simplify the Interface: Combining modules can lead to a simpler, more unified interface, potentially eliminating intermediate interfaces or allowing functionality to be performed automatically (e.g., buffering in file I/O).
Bring Together to Eliminate Duplication: If the same code pattern repeats, factor it out into a separate method or refactor the code so the snippet is executed in one place.
Separate General-Purpose and Special-Purpose Code: A general-purpose mechanism should not include code specialized for a particular use; specialized code should reside in different modules.
Splitting and Joining Methods:
- Length is Not the Only Criterion: Long methods aren't inherently bad if they have simple signatures and are easy to read. Excessive splitting introduces unnecessary interfaces and separates related code.
- Splitting by Factoring Out Subtask: Best when a subtask is cleanly separable and could be used by other methods.
- Splitting into Separate Methods: Makes sense if the original method had an overly complex interface doing multiple unrelated things. Each new method should have a simpler interface, and most callers should only need one of the new methods.
- Joining Methods: Can replace shallow methods with deeper ones, eliminate duplication, remove dependencies, or simplify interfaces.
A Different Opinion: Clean Code: Robert Martin advocates for extremely short functions (even 10 lines is too long), using method names as comments. Ousterhout argues that this often leads to numerous shallow functions, increasing interfaces and making it harder to understand how they work together. Depth is more important than length.

Chapter 10: Define Errors Out Of Existence

Exceptions and Complexity: Exception handling is a major source of complexity. Code dealing with special conditions is inherently harder to write and test, and developers often define exceptions without considering their handling.
Problems with Exceptions:
- Disrupt normal control flow, requiring complex recovery or state restoration.
- Can create secondary exceptions during recovery (e.g., resending a delayed packet, corrupted redundant copy).
- Verbose and clunky language support makes code hard to read and trace.
- Difficult to test, as uncommon exceptions are hard to generate, leading to undetected bugs that surface rarely but catastrophically.
Too Many Exceptions: Programmers often define unnecessary exceptions out of an "over-defensive" style, leading to a proliferation of exceptions that complicate system.
- Punting Problems: Throwing exceptions to avoid difficult situations passes the complexity to the caller, who often faces the same difficulty in handling it.
- Interface Complexity: Exceptions are part of a class's interface and add complexity, making classes shallower.
Techniques to Reduce Exception Handling Complexity:
- Define Errors Out Of Existence: The best way is to modify API semantics so that certain error conditions are no longer exceptional.
  - Java substring Method: The Java substring throwing IndexOutOfBoundsException is unnecessary. It should automatically adjust indices to the string bounds and return the overlapping portion, defining the exception out of existence and simplifying usage.
  - Windows vs. Unix File Deletion: Windows prohibits deleting open files (an error), while Unix marks them for deletion and allows open processes to continue access, delaying the actual deletion until all references are closed, thus defining the "file in use" error out of existence.
  - Catching Bugs vs. Complexity: While exceptions can catch some bugs, they also increase complexity, leading to other bugs. Simpler software with fewer error conditions generally has fewer bugs.
- Mask Exceptions: Handle exceptional conditions at a low level in the system so higher levels are unaware.
  - TCP Example: TCP masks packet loss by retransmitting internally.
  - NFS Example: NFS masks server crashes by retrying requests, causing applications to hang but allowing them to seamlessly resume when the server recovers, avoiding cascading application failures.
- Exception Aggregation: Handle many exceptions with a single piece of code.
  - Web Server Parameter Handling: Instead of multiple try-catch blocks for missing parameters, a single top-level handler in the dispatcher can catch all such exceptions and generate a generic error response, with specific error messages included in the exception object. This approach encapsulates knowledge effectively.
- Just Crash?: For errors that are difficult/impossible to handle and occur infrequently (e.g., out of memory, disk hard errors), the simplest approach is to print diagnostic information and abort the application. This avoids adding significant complexity for rare, unrecoverable situations.
Taking it Too Far: Defining away or masking exceptions only makes sense if the exception information is not needed outside the module. If it's essential for robustness (e.g., knowing about network failures in a communication module), then the exceptions must be exposed.

Chapter 11: Design it Twice

The Principle: It's unlikely that the first design idea will be the best one; considering multiple options for each major design decision leads to better results.
Process:
- Sketch out a few radically different design possibilities, focusing on the most important aspects (e.g., a class's interface).
- Even if one approach seems superior, explore a second to understand its weaknesses and learn from the comparison.
- List pros and cons for each alternative, prioritizing ease of use for higher-level software. Consider interface simplicity, generality, and efficiency.
- The best design might combine features from different alternatives or be an entirely new solution driven by identified problems.

Chapter 12: Why Write Comments? The Four Excuses

Importance of Comments: In-code documentation is crucial for understanding a system, working efficiently, defining abstractions, hiding complexity, and improving design.
Common Excuses for Not Writing Comments (and Rebuttals):
- "Good code is self-documenting.": A myth. While good naming helps, much design information (informal interface aspects, rationale, usage constraints) cannot be expressed in code and requires comments. Without comments, abstractions are lost, forcing developers to read implementation details.
- "I don't have time to write comments.": This conflicts with the "investment mindset." Good comments significantly improve maintainability and quickly pay for themselves. Writing comments for abstractions early on (as part of design) also improves the overall design. Commenting typically adds no more than 10% to development time.
- "Comments get out of date and become misleading.": While true, it's a manageable problem. Updating comments doesn't require enormous effort for large code changes. Organizing documentation to avoid duplication and keeping comments near relevant code, along with code reviews, helps maintain accuracy.
- "All the comments I have seen are worthless; why bother?": This highlights a problem with how comments are written, not their inherent value.
Benefits of Well-Written Comments:
- Capture information from the designer's mind that code cannot express, aiding future understanding and modification.
- Reduce cognitive load by providing necessary information and allowing readers to ignore irrelevant details.
- Reduce "unknown unknowns" by clarifying system structure and relevant code/information for changes.
- Clarify dependencies and eliminate obscurity.
A Different Opinion: Robert Martin's "Comments are Failures": Martin argues comments compensate for code's lack of expressiveness and are "failures." Ousterhout disagrees, stating comments provide information different from code (e.g., abstractions) and are essential for defining abstractions and managing complexity. Replacing comments with short, named methods often leads to cryptic code.

Chapter 13: Comments Should Describe Things that Aren’t Obvious from the Code

Guiding Principle: Comments should describe information that cannot be easily deduced or is not immediately obvious from the code itself. This includes low-level details (e.g., boundary conditions, units), rationale, rules, and especially abstractions.
Why Code Alone Is Insufficient for Abstractions: Code is too low-level and includes implementation details that should be hidden in an abstraction. Comments are necessary to provide the simplified, higher-level view that users need.
Conventions for Commenting: Adopt conventions for what to comment and formatting (e.g., Javadoc, Doxygen) to ensure consistency and encourage actual comment writing.
Don't Repeat the Code: Avoid comments that merely restate what is obvious from the code. Such comments are unhelpful and reinforce the idea that comments are worthless. Use different words in comments than in names to provide additional meaning.
Lower-Level Comments Add Precision: These comments clarify exact meanings and are useful for variable declarations, specifying units, boundary conditions (inclusive/exclusive), null implications, resource ownership, and invariants. They fill in details missing from names/types.
Higher-Level Comments Enhance Intuition: These comments are more abstract, omit details, and help readers understand the overall intent and structure (the "what" and "why," not the "how"). Useful for blocks of code and interface comments.
- Explaining "Why": Comments can explain the rationale or conditions under which code is executed (e.g., "how we get here").
Interface Documentation: Essential for defining abstractions. It should be separated from implementation comments.
- Class Interface Comment: High-level description of the class's abstraction, capabilities, and instance representation, possibly limitations.
- Method Interface Comment: Describes behavior for callers, arguments, return value, side effects, exceptions, and preconditions. Should provide all information needed to invoke without reading implementation.
Implementation Comments: What and Why, Not How: Focus on what code blocks are doing (abstract description) and why (tricky aspects, bug fixes), rather than how they work. Most short, simple methods don't need them.
Cross-Module Design Decisions: Document complex design decisions affecting multiple classes in a central, discoverable place (e.g., a designNotes file with short in-code references).
Comments Belong in the Code, Not the Commit Log: Detailed information about code changes and their motivations should be in the code itself, not just commit messages. Commit logs are not easily discoverable or browsable for ongoing development.
Conclusion: Comments ensure code is obvious to readers, filling in information not apparent from the code.

Chapter 14: Choosing Names

Importance of Good Names: Good names are a form of documentation; they make code easier to understand, reduce the need for other documentation, and help detect errors. Poor names increase complexity, ambiguities, and bugs.
Complexity is Incremental (Naming): While one mediocre name won't destroy a system, thousands of bad names significantly impact complexity and manageability.
Create an Image: The goal of naming is to create a clear image in the reader's mind about the entity's nature, what it is, and what it is not. Names are a form of abstraction.
Names Should Be Precise:
- Avoid Vagueness/Generality: Generic or vague names (e.g., "count," "x," "y," "status," "result") obscure meaning and lead to misuse.
- Use Predicates for Booleans: Boolean variable names should clearly indicate true/false meaning (e.g., cursorVisible).
- Avoid Ambiguous Similarities: Names that are too similar (e.g., struct socket and struct sock in Linux kernel) cause confusion.
- Exceptions for Short Loops: Generic names like i and j are acceptable for short loop iteration variables where their scope is immediately visible.
- Avoid Over-Specificity: A name being too specific limits the perceived use of a general-purpose entity.
- Red Flag: Hard to Pick Name: Difficulty in finding a precise, intuitive, and concise name suggests a poor design or unclear purpose for the underlying entity.
Use Names Consistently: For frequently used concepts, establish a consistent naming convention and stick to it. This reduces cognitive load and allows for safe assumptions.
Avoid Extra Words: Every word should add useful information. Avoid generic nouns (e.g., "object," "field"). Also, don't repeat the class name in an instance variable name within that class.
Different Opinion: Go Style Guide: The Go language advocates very short names, sometimes single characters, arguing that long names obscure code. Ousterhout finds this can lead to ambiguity and requires more mental effort to deduce meaning, preferring names that provide clearer clues.

Chapter 15: Write The Comments First (Use Comments As Part Of The Design Process)

Delayed Comments are Bad Comments: Postponing comment writing until after coding and testing leads to poor quality documentation because it's often never written, or written hastily when enthusiasm for the project has waned and design details are fuzzy.
Write the Comments First Approach:
- Start with the class interface comment.
- Write interface comments and signatures for important public methods (leave bodies empty).
- Iterate on these comments until the basic structure feels right.
- Write declarations and comments for important class instance variables.
- Fill in method bodies, adding implementation comments and new method/variable comments as needed.
- When code is done, comments are done.
Benefits of Comments-First:
- Better Comments: Key design issues are fresh in mind, leading to more accurate and complete comments. Focus on abstraction before implementation details.
- Comments as a Design Tool: The most important benefit. Writing comments forces identification of the essence of a variable or code, which is critical for good design. It allows for early evaluation and tuning of abstractions.
- Canary in the Coal Mine of Complexity: A long or complicated comment for a method or variable is a "red flag" (Hard to Describe) indicating a shallow class, complex interface, or poor variable decomposition. This helps identify and fix design problems early.

Chapter 16: Modifying Existing Code

Software Evolution: Software systems evolve iteratively, with designs constantly changing. The design of a mature system is more a product of its evolution than initial conception.
Stay Strategic: When modifying existing code (bug fixes, new features), avoid the "smallest possible change" tactical mindset. This introduces special cases and complexity.
Strategic Approach to Modification: Aim for the system to have the structure it would have had if the change was considered from the start. Refactor and improve the design with every modification. This is an investment that speeds up future development.
Addressing Constraints: While tight deadlines or broad incompatibilities might necessitate quick fixes, always seek cleaner alternatives or plan for future refactoring. Organizations should allocate time for cleanup and refactoring.

Chapter 17: Consistency

Definition: Similar things are done in similar ways, and dissimilar things are done in different ways.
Benefits:
- Cognitive Leverage: Once a pattern is learned, that knowledge can be reused to quickly understand other similar situations, reducing cognitive load.
- Reduces Mistakes: Prevents incorrect assumptions based on familiar-looking but inconsistent patterns, making assumptions safer.
Examples of Consistency:
- Names: Using names consistently.
- Coding Style: Following style guides for indentation, brace placement, naming, commenting, etc., improves readability and reduces errors.
- Interfaces: Multiple implementations of the same interface reduce cognitive load because the common interface is already known.
- Design Patterns: Using established solutions to common problems makes code more familiar and easier to understand.
- Invariants: Properties of variables/structures that are always true simplify reasoning about code behavior and reduce special cases.
Ensuring Consistency:
- Document: Create and maintain a document listing important overall conventions (e.g., style guides). Place it conspicuously and encourage team members to read it. For localized conventions (e.g., invariants), document them in the code.
- Enforce: Implement automated tools (e.g., pre-commit scripts) to check for and prevent violations of syntactic conventions. Code reviews also help enforce conventions and educate developers.
- "When in Rome...": Developers should follow existing conventions within a file or project. Look for existing patterns and mimic them.
- Don't Change Existing Conventions: Resist the urge to "improve" on existing conventions unless there is significant new information and the new approach is demonstrably superior enough to warrant updating all old uses. Inconsistency is almost always worse than a slightly less optimal but consistent approach.
Taking it Too Far: Consistency applied to dissimilar things creates complexity and confusion. Consistency benefits only when developers can confidently assume "if it looks like an x, it really is an x."

Chapter 18: Code Should be Obvious

Definition of "Obvious" Code: Code is obvious if a reader can understand its behavior and meaning quickly, without much thought, and their initial guesses are correct.
Reader's Perspective: "Obvious" is in the mind of the reader. Code reviews are crucial for identifying nonobvious code. If a reviewer finds it nonobvious, it needs clarification.
Things That Make Code More Obvious:
- Good Names: Precise and meaningful names clarify code and reduce documentation needs. Vague names force readers to deduce meaning.
- Consistency: Similar things done in similar ways allow readers to recognize patterns and draw safe conclusions quickly.
- Judicious Use of White Space: Formatting (e.g., blank lines, indentation) improves readability by clarifying structure and making comments more visible.
- Comments: When code cannot be made obvious, comments are essential to provide missing information and clarify confusion.

Chapter 19: Software Trends

Object-Oriented Programming and Inheritance:
- Benefits: Mechanisms like classes, private methods, and instance variables can help achieve better designs (e.g., information hiding through private elements).
- Interface Inheritance: A parent class defines method signatures without implementation. Subclasses provide different implementations. This reduces complexity by reusing interfaces for multiple purposes, increasing an interface's "depth."
- Implementation Inheritance: A parent class provides default implementations that subclasses can inherit or override. Reduces code duplication across subclasses, minimizing change amplification.
- Drawbacks of Implementation Inheritance: Creates dependencies between parent and subclasses (e.g., parent's instance variables accessed by children), leading to information leakage. Modifying one class in the hierarchy often requires examining others, increasing complexity. Should be used with caution; composition may be a better alternative.
Agile Development:
- Focus: Primarily a process for software development (teams, schedules, testing, customer interaction), rather than design principles.
- Incremental and Iterative Development: Emphasizes developing in series of iterations, adding and evaluating features. This aligns with the book's advocacy for incremental design where abstractions are refined over time.
- Risk: Tactical Programming: Agile can lead to tactical programming by focusing on features over abstractions and encouraging delayed design decisions. This can result in rapid complexity accumulation.
- Ousterhout's Stance: Increments of development should be abstractions, not just features. Design abstractions cleanly and comprehensively when needed, rather than in small pieces over time.
Unit Tests:
- Shift in Practice: Programmers now widely write unit tests, which are small, focused tests for single methods or small code sections.
- Facilitates Refactoring: Unit tests are crucial for enabling refactoring. A robust test suite provides confidence that structural changes won't introduce undetected bugs, encouraging developers to make design improvements.
- High Code Coverage: Unit tests offer better code coverage than system tests, making them more effective at finding bugs.
Test-Driven Development (TDD):
- Process: Write unit tests before writing code; then write just enough code to make tests pass.
- Critique: Ousterhout is not a fan. TDD focuses on getting features working, leading to tactical programming and potentially messy designs with no clear time for broader design.
- Alternative: Design abstractions comprehensively when needed, rather than incrementally through tests.
- When TDD Makes Sense: Useful for fixing bugs; write a failing unit test first, then fix the bug to make the test pass.
Design Patterns:
- Purpose: Commonly accepted solutions for particular problems (e.g., iterator, observer).
- Benefits: Generally provide clean solutions to common problems, making implementation faster and more reliable.
- Risk: Over-application: Not every problem fits an existing pattern. Forcing a problem into an ill-fitting pattern can lead to more complexity than a custom approach. "More design patterns are not necessarily better."
Conclusion: Always challenge new software development paradigms from the standpoint of complexity, as some may inadvertently worsen complexity rather than improve it.

Chapter 20: Designing for Performance

Goal: Achieve high performance without sacrificing clean design. Simplicity usually makes systems faster.
How Much to Worry About Performance: Avoid optimizing every statement (slows development, adds complexity, often ineffective). Also avoid completely ignoring it (leads to "death by a thousand cuts" with widespread inefficiencies). The best approach is to use basic knowledge of expensive operations to choose naturally efficient and simple designs.
Fundamentally Expensive Operations: Network communication, I/O to secondary storage, dynamic memory allocation, cache misses.
Learning Expensive Operations: Use micro-benchmarks to measure the cost of single operations in isolation.
Complexity vs. Performance Trade-off: If efficiency adds only minor, hidden complexity without affecting interfaces, it might be worthwhile. If it adds significant complexity or complicates interfaces, start with simpler design and optimize later if needed, unless performance is clearly critical from the outset.
Measure Before (and After) Modifying: Always measure system behavior before making performance tweaks; programmers' intuitions are unreliable. Measurements identify bottlenecks and provide a baseline to confirm improvements. If no measurable difference, revert changes.

Chapter 21: Decide What Matters

Core Principle: Separate what matters from what doesn't. Structure software systems around the important things, minimizing the impact of less important things. Emphasize what matters (make it obvious), and hide what doesn't.
Relevance to Earlier Concepts: This principle is fundamental to abstraction (interface reflects what matters to users), naming (names convey essential information).
How to Decide What Matters:
- Leverage Points: Look for solutions that solve multiple problems or pieces of information that clarify many other things (e.g., general-purpose interfaces, invariants).
- Multiple Options ("Design it Twice"): Comparing different options helps identify what is truly most important.
- Hypothesis and Learning: For less experienced developers, form a hypothesis about what matters, build based on it, and learn from the outcome (whether right or wrong).
Emphasize Things That Matter:
- Prominence: Place important things where they are easily seen (interface documentation, names, heavily used method parameters).
- Repetition: Key ideas should appear repeatedly.
- Centrality: Most important ideas should form the core structure of the system (e.g., device driver interfaces).
- Conversely, deemphasize what doesn't matter by hiding it and ensuring it doesn't impact system structure or frequent encounters.
Mistakes:
- Treating Too Many Things as Important: Clutters design, adds complexity, increases cognitive load (e.g., irrelevant method arguments, forcing awareness of unnecessary distinctions like buffered I/O, shallow classes).
- Failing to Recognize Importance: Hides crucial information or functionality, impedes productivity, and leads to "unknown unknowns."
Broader Application: The principle applies beyond software design to technical writing (structuring around key concepts) and life philosophy (focusing energy on what truly matters).
"Good Taste": The ability to distinguish what is important from what is not important is a key attribute of a good software designer.

Chapter 22: Conclusion

Central Theme: The book's focus is entirely on managing complexity, which is the most significant challenge in software design, affecting buildability, maintainability, and performance.
Key Takeaways:
- Understanding the root causes of complexity.
- Recognizing "red flags".
- Applying general design principles.
- Adopting an "investment mindset" for design.
Payoff of Good Design: Investments in good design pay off quickly through reusable modules, clearer documentation, and improved design skills, ultimately leading to faster development of higher-quality software and a more enjoyable process.

Modern Software Engineering Chapter 4–8 Summary

Mark Adel — Sat, 29 Mar 2025 21:54:31 +0000

I recently finished reading Modern Software Engineering by Dave Farley and would like to share summaries of the five chapters I found most useful, especially for new software engineers. In my opinion, reading these summaries can serve as a substitute for reading the full chapters. I generated each summary using ChatGPT and then refined them through several iterations to ensure they capture all the main ideas accurately.

Chapter 4 (Working Iteratively)

Iteration is a procedure that drives learning. Iteration allows us to learn, react, and adapt to what we have learned. Without iteration, and the closely related activity of collecting feedback, there is no opportunity to learn on an ongoing basis. Fundamentally, iteration allows us to make mistakes and to correct them, or make advances and enhance them.
Iteration allows us to progressively approach some goal. Its real power is that it allows us to do this even when we don't really know how to approach our goals. As long as we have some way of telling whether we are closer to or further from our goal. We could even iterate randomly and still achieve our goal. We can discard the steps that take us further away and prefer the steps that move us nearer. This is in essence how evolution works. It is also at the heart of how modern machine learning (ML) works.
We can't afford to spend lots of time in analysis and design without creating anything, because that means more time not learning what really works. We need to do just enough analysis, design, coding, testing, and releasing to get our ideas out into the hands of our customers and users so that we can see what really works. We need to reflect on that and then, given that learning, adapt what we do next to take advantage of it.
Industry data says that for the best software companies in the world, two-thirds of their ideas produce zero or negative value. We are terrible at guessing what our users want. Even when we ask our users, they don't know what they want either. The most effective approach is to iterate. It is accepting that some, maybe even many, of our ideas will be wrong and work in a way that allows us to try them out as quickly, cheaply, and efficiently as possible.
To make progress we must take a chance, make a guess, be willing to take a risk. We are very bad at guessing, though, so we need to work more defensively by proceeding in small steps and limit the scope of our guesses.
If we work iteratively in small steps, the cost of any single step going wrong is inevitably lower.
An iterative approach allows us to always have the most up-to-date picture of the situation that we are really in, rather than some predictive, theoretical, always-inaccurate version of that situation. It allows us to learn, react, and adapt as changes happen along the way. Working iteratively is the only effective strategy for a changing situation.
Practices like Continuous Integration and Continuous Delivery are inherently iterative, promoting frequent, small changes and continuous learning. This approach not only improves the quality of the code but also aligns development practices with the iterative mindset.

Chapter 5 (Feedback)

Without feedback, there is no opportunity to learn. We can only guess, rather than make decisions based on reality. Despite this, it is surprising how little attention many people and organizations pay to it.
Software development is always an exercise in learning, and the environment in which it takes place is always changing; therefore, feedback is an essential aspect of any effective software development process.
By following an approach like test-driven development, where tests are written and run before coding, fast, high-quality feedback can be achieved in milliseconds. This enables immediate validation of correctness, quickly catching mistakes and design issues.
Continuous integration is about evaluating every change to the system along with every other change as frequently as possible, as close to "continuously" as we can practically get. Continuous integration, when practiced correctly, means that we get regular, frequent drips of feedback. It gives us powerful insight into the state of our code and the behavior of our system throughout the working day. For CI to work, we have to commit our changes frequently enough to gain that feedback. Not only does this approach mean that our software is always in a deployable state, but it also encourages us to design our work in a way that sustains this approach.
A feedback-driven approach also shapes software architecture. Continuous delivery demands that software be constantly ready for release. This encourages architectures that are modular, testable, and easier to deploy.
Feedback loops in product design validate the effectiveness of ideas by directly measuring customer usage and satisfaction. Continuous delivery supports this by allowing organizations to deploy and test ideas quickly, enabling faster learning about customer needs and product effectiveness.

Chapter 6 (Incrementalism)

If working iteratively is about refining and improving something over a series of iterations, then working incrementally is about building a system, and ideally releasing it, piece by piece.
To create complex systems, we need both approaches. An incremental approach allows us to decompose work and to deliver value step-by-step (incrementally), getting to value sooner and delivering value in smaller, simpler steps.
Working in ways that allow us the freedom to change our code and change our minds as our understanding deepens is fundamental to good engineering and is what incrementalism is built upon. Striving to be able to work incrementally then is also striving for higher-quality systems. If your code is hard to change, it is low quality, whatever it does.

Chapter 7 (Empiricism)

Empiricism focuses on decision-making based on evidence and real-world observations. In engineering, especially software engineering, empiricism ensures that solutions are grounded in practical reality rather than abstract theories.
Software systems often surprise developers when deployed in production. These surprises should be seen as opportunities to learn. Production environments expose the reality of a system's behavior, making it clear that software is a "best guess" at functionality. The true test of a system's quality comes in real-world usage, where unexpected behaviors are inevitable.
While both experimentation and empiricism rely on observation, they serve different purposes. Experimentation tests a specific hypothesis under controlled conditions, while empiricism involves observing outcomes from real-world environments without the strict structure of a controlled experiment. Empiricism can validate or challenge assumptions based on broader, more informal observations. It is not a replacement for structured experimentation but rather complements it by allowing developers to observe results in less controlled but more realistic settings.
Empiricism is essential in debugging and problem-solving. Assumptions based on prior experience can lead to incorrect conclusions, and only by gathering real-world evidence and thoroughly testing can the true nature of an issue be uncovered. Empirical investigation, rather than reliance on intuition or preconceived theories, leads to more accurate understanding and resolution of complex problems in software systems. This highlights the need to challenge assumptions and base decisions on observable facts.
Human beings are prone to cognitive biases and shortcuts, often making us jump to conclusions based on incomplete data. These biases helped humans survive in primitive environments but can lead to faulty decisions in complex engineering problems. To avoid self-deception, it is crucial to gather organized, structured feedback from reality and resist the temptation to shape facts to fit preconceived theories. Richard Feynman famously characterized science as follows: The first principle is that you must not fool yourself – and you are the easiest person to fool. Empiricism requires rigorous attention to facts and a willingness to challenge initial impressions.
Developers may invent realities that suit their assumptions, particularly when working with complex systems. Theoretical models that seem correct in principle often fail when tested empirically.
Empirical reality must guide decision-making, not assumptions or untested theories. Skepticism and evidence-based problem-solving should be central to development practices. Developers should assume their assumptions might be wrong and actively seek evidence to confirm or refute them.
Engineering, unlike pure science, requires us to focus on the practicality of our solutions. This is where empiricism becomes crucial. It's not sufficient to simply observe the world, make assumptions based on those observations, and assume we are correct just because the data came from real-world sources. That approach leads to both bad science and poor engineering. Since engineering is a practical field, we must remain skeptical of our assumptions and continually test them through experiments, always verifying them against real-world experience.

Chapter 8 (Being Experimental)

Being experimental means moving away from decisions based on authority and instead basing decisions on empirical evidence. It encourages a shift in mindset where decisions in software development are made based on experiments and real-world evidence. For example, instead of debating whether Clojure is better than C#, small trials can test aspects like stability and performance to make informed choices.
Feedback is central to being experimental in software development. Efficient, frequent feedback enables developers to quickly learn from the results of their work and adjust their approach. Whether the feedback comes from automated tests or user interactions, it must be gathered quickly and interpreted carefully to drive continuous improvement.
Every experiment starts with a hypothesis—a proposed explanation or prediction that can be tested. In software engineering, hypotheses might be about the effectiveness of a certain architecture, the scalability of a system, or the speed of a specific code optimization. Hypotheses should be clear, specific, and measurable.
Measurement is crucial for validating the predictions of hypotheses. However, poor measurements can mislead, as illustrated by a client case where incentivizing developers based on test coverage led to the creation of useless tests with no assertions. This example shows how focusing on the wrong metrics can skew results. The correct measurement in this case would have been software stability, not just the number of tests. A good experiment relies on accurate, meaningful metrics that reflect the true quality or effectiveness of the software.
Controlling variables is necessary to gather reliable data from experiments. By controlling variables (like environment, input date, dependencies, hardware, etc.), you ensure that any changes in the outcome can be attributed to the variable you're testing — not something else. For example, if you're testing a new caching strategy, but the server load is fluctuating wildly, your results might be skewed. Controlling for load ensures the cache strategy is the real difference-maker.
Automated tests are considered small-scale experiments that provide constant feedback on software quality. They can be seen as hypotheses about how the software should behave under certain conditions, and running these tests validates or invalidates the predictions.
Different sciences, such as physics or biology, might have different degrees of precision, but they all benefit from systematic experimentation. This analogy is used to reinforce that while software experiments might not reach the precision of physics, they are still valuable for learning and improving. By organizing software development around experiments, teams can generate deeper insights into system behavior and make better decisions.

Make It Work, Then Make It Better

Mark Adel — Wed, 27 Nov 2024 22:11:57 +0000

In software engineering, "Make it work, then make it better" is a pragmatic approach to development. It stresses the importance of first building a working solution before refining and optimizing it. It consists of two phases:

The first phase focuses on ensuring the software works as intended, delivering the expected results and meeting the requirements. At this stage, we make the "computer" understand the code.
The second phase shifts the focus to humans, refactoring the code for your future self and fellow developers who will read, use, and maintain it.

This doesn't mean ignoring code quality in the first phase; it might take around 30% of the focus, depending on context.

Needless to say, phase two should always be completed before submitting the code for review or delivering it to customers.

Make It Work

Prioritizing Functionality: Ensuring the program works as intended. This might involve hardcoded values, inefficiencies, unhandled exceptions, etc., but that's okay for now.

Speed Over Perfection: Delivering a working solution quickly allows us to validate our approach before investing time in enhancing it.

Understanding the Problem: Writing quick, functional code helps developers gain a better understanding of the problem's details.

Make It Better

Here are some key areas to focus on:

Clear Naming: Assigning clear and descriptive names to variables, functions, and classes that reflect their intent.

Code Style: Adhering to coding standards and guidelines to ensure consistency and readability.

Code Design: Following established design principles like SOLID to enhance code maintainability.

Architecture: Revisiting the architecture decisions to ensure they suit the specific feature requirements.

Testing: Writing automated tests of the appropriate type (unit, integration, etc.) to ensure the code behaves as expected and prevents new updates from breaking existing functionality.

Logging: Adding meaningful logs to improve visibility and aid debugging.

Error Handling: Ensuring exceptions are handled correctly and gracefully to maintain stability and reliability.

Comments: Adding comments where necessary to clarify non-obvious logic.

Performance: Ensuring that the code is efficient and scalable as needed.

Conclusion

"Make it work, then make it better" encourages developers to focus on solving problems pragmatically before striving for perfection. Write the messy first draft, but don't forget to come back and polish it into a masterpiece!

It's very common for the second phase to be ignored altogether, and this is exactly how technical debt accumulates. Always make sure to apply the second phase when building software, unless it's a throwaway prototype. This at least ensures that the software quality will not degrade over time, which in my experience is an achievement in itself.

It's worth noting that this principle isn't limited to programming; you can apply it to many areas of life. In fact, I used this approach while writing this blog! Here's how it went:

I wrote a quick draft. (phase 1)
I used chatGPT to do some enhancements and rephrasings. (phase 2)
I made several iterations on chatGPT's output to make it simpler and more relevant. (phase 2)

Fixing VS Code Go to Definition Issue on Linux Mint

Mark Adel — Sat, 19 Oct 2024 13:57:45 +0000

After switching from Linux Ubuntu to Mint (Cinnamon) on my work laptop, I discovered that the Go to Definition functionality in Visual Studio Code, which uses Alt+Click, had stopped working.

This issue had been driving me crazy for weeks until I finally fixed it.

The problem was that Mint, by default, uses Alt+Click to move and resize windows, which was overriding and conflicting with VS Code’s Go to Definition functionality.

Solution

In Mint’s Windows Settings, change the “Special key to move and resize windows” from <Alt> to <Super>.

Logging Done Right

Mark Adel — Sun, 09 Jun 2024 02:51:30 +0000

Writing effective log messages is crucial for the overall observability of your application. In this guide, we are going to focus mainly on what to log, and how to write effective log messages.

The code examples are written in JavaScript for its simple syntax, but these guidelines are applicable to any programming language.

Let's get started!

1. All log messages should start with the class and function name

logger.info('[MyClass.myFunction] The log message');

Reason: Quickly identifying where the log message comes from. It also adds uniqueness to the logs, ensuring that logs are not duplicated.

2. Add logs in all exception handling blocks

try {
    // code that might throw an error
} catch (error) {
    logger.error('[OrderService.placeOrder] Order placement failed: ' + error.message);
}

Reason: Very crucial for troubleshooting.

3. Include context if useful

logger.warn('[PaymentGateway.processPayment] Payment failed for UserID: 123, OrderID: 456, Error: Insufficient funds');

Reason: Context helps you understand the conditions under which the log was generated, aiding in replicating and fixing issues.

4. Add logs for auditing

logger.info('[OrderService.placeOrder] Order ID: 12345 placed successfully');

Reason: Audit logs have many benefits, such as reconstructing the timeline of a system outage or a security breach, and also being able to detect them while happening.

5. Don't include large log messages if not necessary

Bad:

logger.info('[SomeOrdersJob.processOrders] Finished processing orders chunk, index: ' + chunkIndex + 'orders: ' + JSON.stringify(orders));

Good:

logger.info('[SomeOrdersJob.processOrders] Finished processing orders chunk, index: ' + chunkIndex + 'orders length: ' + orders.length);

Reason: Large log messages reduce readability, consume more space, and might introduce performance implications.

6. Don't log sensitive data

Bad, too bad:

logger.warn('[AuthService.login] User login attempt failed, login data: ' + JSON.stringify(loginData));

logger.warn('[PaymentGateway.registerPaymentCard] Card registration failed, card data: ' + JSON.stringify(cardData));

Good:

logger.warn('[AuthService.login] User login attempt failed, username: ' + loginData.username);

Reason: Logging sensitive data can create security risks and violate privacy regulations such as GDPR. Always sanitize data before logging. It's very likely to overlook sanitizing your data when logging HTTP requests and responses, for example.

7. Use the correct log level

logger.debug('[OrderService.placeOrder] Inside placeOrder function');
logger.info('[OrderService.placeOrder] Order ID: 12345 placed successfully');
logger.warn('[AuthService.login] User login attempt failed, username: ' + loginData.username);
logger.error('[OrderService.placeOrder] Order placement failed: ' + error.message);
logger.fatal('[Database.connect] Unable to connect to the database: ' + error.message);

Reason: Using log levels has many benefits, such as filtering logs based on their level, taking specific actions such as sending an alert for high-severity logs.

8. Timestamp your logs

logger.info('[2024-06-08T12:00:00Z] [OrderService.placeOrder] Order ID: 12345 placed successfully');

Reason: Timestamps provide a chronological order to logs, making it easier to track and understand the sequence of actions. They also allow you to troubleshoot issues that happened at specific times.

9. Avoid logging in loops or recursive functions if not necessary

Bad:

for (let i = 0; i < items.length; i++) {
    logger.info('[InvoiceCalculator.calculate] Calculating item: ' + i);
    // calculation logic
}

Good:

logger.info('[InvoiceCalculator.calculate] Calculation started');
for (let i = 0; i < items.length; i++) {
    // calculation logic
}
logger.info('[InvoiceCalculator.calculate] Calculation finished');

Reason: Excessive log entries make it harder to find relevant information, consume more space, and can lead to performance issues.

10. Log the start and end of long-running operations

logger.info('[DataImportService.importData] Data import started');

// long-running operation

logger.info('[DataImportService.importData] Data import completed');

Reason: It helps you monitor the progress and duration of these operations.

11. Ensure log messages are concise and clear

Bad:

logger.info('[OrderService.placeOrder] The order placement function has successfully completed processing the order with ID 12345');

Good:

logger.info('[OrderService.placeOrder] Order ID: 12345 placed successfully');

Reason: Concise and clear log messages are easier to read and understand.

Conclusion

I hope you found this guide helpful. Please feel free to suggest other practices that you think are important, and I will be happy to include them.

References and Further Reading

Disaster Recovery: Better Safe Than Sorry

Mark Adel — Fri, 26 Apr 2024 17:20:58 +0000

Do you work for a tech startup? Does your startup have a disaster recovery plan? If not, read on.

What is disaster recovery?

Disaster recovery (DR) is an organization's ability to restore access and functionality to IT infrastructure after a disaster event, whether natural or caused by human action (or error). DR is considered a subset of business continuity, explicitly focusing on ensuring that the IT systems that support critical business functions are operational as soon as possible after a disruptive event occurs. 1

As someone who has experience working in fast-paced tech startups. I know that focusing on rapid growth can sometimes lead to overlooking critical operational aspects, such as having a disaster recovery plan (DRP).

There are several reasons why a startup might ignore or overlook having a DRP, including:

Constantly prioritizing shipping features that lead to immediate gains.
Lack of awareness about the consequences of not having one.
The startup hasn't yet faced its first major disaster, leading to a false sense of security.
Resistance to investing time and resources into planning for hypothetical scenarios.
Lack of relevant expertise or guidance, which leads to procrastination in planning and implementing disaster recovery.
The misconception that disaster recovery is only necessary for large corporations.

None of the reasons above justify not implementing disaster recovery. A simple human error, such as an employee executing a dangerous command, can kill the startup and everything it has been working for over its lifetime. Disaster recovery is an essential aspect of risk management in any organization, regardless of its size.

If you are part of a startup that still doesn't have a DRP, it's important to speak up. Share your concerns, and educate your team and management about the importance of disaster recovery. Your efforts will be greatly appreciated when the company faces a disruptive event and can recover from it with minimal damage.

Essential steps for implementing disaster recovery:

Evaluate potential threats and identify the most critical system components that require protection against disasters.
Create a plan that outlines the following:
- The necessary actions to protect these components, such as preventative measures and data backups.
- The procedures for responding to different types of disasters.
- The individuals responsible for executing these procedures.
Ensure that everyone understands their role during a disaster and knows how to execute the recovery procedures.
Regularly update your backups to minimize the damage caused by data loss during disasters.

Please note that these are the minimum steps. Depending on the nature of your business, you may need to include additional steps or details in your DRP.

Remember, a well-implemented DRP can be the difference between minor damage and a catastrophic failure for your organization.

Conclusion

I hope this post has been an eye-opener. The importance of disaster recovery cannot be overstated. It's not just about surviving a disaster, but also about maintaining trust and ensuring business continuity. A minor effort now can prevent a major crisis later. It's always better to be safe than sorry. Good luck!

Diagnose Before You Prescribe

Mark Adel — Wed, 24 Apr 2024 03:35:05 +0000

Imagine you go to the doctor feeling unwell, and without examining you, they immediately start writing a prescription. This is exactly what some software engineers occasionally do.

When we are faced with an issue in a particular component of our system, especially if it's a performance issue that is challenging to debug, it's not uncommon to find ourselves trying to think of ways to fix it. We may even feel urged to redesign the entire component in a way that we believe is more efficient than the current one, and spare ourselves the burden of debugging.

This will always be the wrong approach. Rushing to solutions without fully understanding the problem can lead to wasted time because there is a high chance that the solution doesn't address the problem. Even if the solution works, you will never be sure whether it really solves the problem or if it's just a coincidence.

Therefore, it's important to resist the urge to jump to solutions and instead, invest time in thoroughly understanding the problem. When faced with a performance issue, the first thing you should do is try to understand the root cause. This involves debugging, examining the logs and relevant performance metrics, etc. Only then should you start looking for a solution, which might be a very simple update that requires minimal effort.

Backend Code Review Checklist

Mark Adel — Sat, 30 Mar 2024 21:39:50 +0000

I have put together this checklist, which I believe will be applicable to most backend code reviews.

Some of these checks, such as Code Style, should ideally be enforced and detected in the CI pipeline. However, I have included them here for the sake of completeness.

You can use this checklist as a starting point and customize it to suit your specific needs.

Code Style

Verify that the code adheres to the agreed-upon coding style guidelines.

Code Maintainability

Verify that the code adheres to the clean code principles (or any other agreed-upon principles).

Requirements

Verify that the code fulfills the specified requirements.
Verify that new code doesn't break any existing functionality.

API Design

Verify that any new APIs adhere to the agreed-upon API design guidelines.

Documentation and Comments

Verify that complex logic or non-obvious decisions are covered by clear comments.
Verify that any required internal or external code documentation is provided, depending on your agreed-upon documentation processes.

Error Handling

Verify that exceptions are handled correctly and that error messages are informative.

Security

Verify that inputs are validated properly.
Verify that sensitive data (passwords, tokens) are securely stored and aren't leaked to logs.
Examine the code for potential security vulnerabilities, such as SQL injection or authentication issues.

Dependencies

Verify that dependencies are up-to-date and don't have known security vulnerabilities.
Verify that any breaking changes are handled when updating dependencies.

Logging

Verify that critical places in the code are covered by logs that are useful for debugging.
Verify that logging adheres to the agreed-upon logging guidelines.

Testing

Verify that the code is covered by the appropriate types of automated tests.

Performance

Evaluate the code for performance issues (memory, CPU, network).
Verify that database queries are optimized.

Version Control

Verify that the agreed-upon version control workflow and practices are followed.

Spelling

Verify that the spelling is correct, as this makes the code more searchable.

Conclusion

I hope you found this checklist useful. Please feel free to suggest additional checks that you think are necessary.

Useful Global Git Configurations

Mark Adel — Wed, 06 Mar 2024 13:31:00 +0000

In this post, I'm going to share with you six global Git configurations that I use and find incredibly useful.

Global Git configurations are settings that apply to all your local repositories. They are stored in a file called .gitconfig in your home directory. You can view and edit this file, or you can use the git config command to modify it from the terminal.

To see the global configurations through the terminal, you can run:

git config --global --list

Let's get to it!

1. `git config --global init.defaultBranch main`

This sets the default branch name to main instead of master when creating a new repository using git init.

2. `git config --global help.autoCorrect prompt`

By default, Git checks for spelling errors and suggests corrections, but it doesn’t automatically apply them. After configuring this setting, Git will prompt you to run the suggested correction.

$ git pusj
WARNING: You called a Git command named 'pusj', which does not exist.
Run 'push' instead [y/N]?

To allow git to run corrections automatically, you can use help.autoCorrect 1 (runs after 0.1 seconds), help.autoCorrect 10 (runs after 10 seconds), or help.autoCorrect immediate (runs immediately).

I prefer using help.autoCorrect prompt to ensure that I will run the command that I intend to.

3. `git config --global branch.sort -committerdate`

This makes git branch sort the branches by the most recently used branches instead of alphabetically. This helps you to quickly find the branches that you are working on.

4. `git config --global fetch.prune true`

When you run git fetch, any branches that have been deleted on the remote repository will automatically be deleted from your local repository.

5. `git config --global log.date iso`

When you run git log, dates will be displayed in ISO format for better readability.

For example, 2024-03-16 05:30:02 instead of Sun Mar 16 05:30:02 2024.

6. `git config --global push.autoSetupRemote true`

This makes git push automatically set up the remote branch. Otherwise, you'd need to do it manually like so:

git push --set-upstream origin <branch-name>

Conclusion

I hope you found these configurations helpful. Please feel free to share yours!

You can find more on Git documentation.

PostgreSQL Isolation Levels and Locking Summary

Mark Adel — Fri, 16 Feb 2024 19:54:27 +0000

I initially prepared this summary for myself. However, I thought other people would find it useful, so I refined it as necessary and decided to share it.

Please note that this serves only as a summary or even a cheat sheet. If you are new to these topics, it's advisable to do further reading.

Read Phenomena

Read phenomena are potential issues (anomalies) that can occur in a database when multiple transactions are executed concurrently.

Dirty Read: A transaction reads data written by another concurrent uncommitted transaction.
- It's like reading a draft that someone else is still writing.
Nonrepeatable Read: A transaction re-reads columns and finds that their values have been changed due to another concurrently committed transaction.
- It's like re-reading a page of a book and finding that the words have changed since your first read.
Phantom Read: A transaction re-reads a set of rows satisfying a specific criteria and finds newly added or removed rows due to another concurrently committed transaction.
- It's like counting a group of people, then finding the number changes when you count again because new people have left or joined.
Serialization Anomaly: The state of data resulting from executing transactions concurrently differs from the state resulting from running them one at a time.
- It's like when you are baking a cake, it might taste differently if you added all the ingredients at the same time than if you added them one at a time in a specific order.

Isolation Levels

Isolation levels determine how transactions influence each other.

`Read Uncommitted`

Dirty Read: Possible, but not in PostgreSQL.
Nonrepeatable Read: Possible.
Phantom Read: Possible.
Serialization Anomaly: Possible.

READ UNCOMMITTED behaves the same as READ COMMITTED in PostgreSQL.

`Read Committed`

Dirty Read: Not possible.
Nonrepeatable Read: Possible.
Phantom Read: Possible.
Serialization Anomaly: Possible.

READ COMMITTED is the default isolation level in PostgreSQL.

`Repeatable Read`

Dirty Read: Not possible.
Nonrepeatable Read: Not possible.
Phantom Read: Possible, but not in PostgreSQL.
Serialization Anomaly: Possible.

`Serializable`

Dirty Read: Not possible.
Nonrepeatable Read: Not possible.
Phantom Read: Not possible.
Serialization Anomaly: Not possible.

You can set the transaction isolation level in PostgreSQL using the following syntax:

BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
-- Your queries
COMMIT;

Locking (Row-level locking)

Locking is a mechanism that prevents concurrent transactions from interfering with each other when accessing or modifying the same data.

One of the primary use cases for locking is to prevent anomalies such as the Lost Update, which can occur in lower isolation levels such as READ COMMITTED.

The Lost Update anomaly occurs when two transactions attempt to update the same data concurrently, and the changes made by one transaction are overwritten by the other, resulting in the loss of the first transaction updates.

There are two common approaches to locking: pessimistic locking and optimistic locking.

Pessimistic Locking

Pessimistic locking assumes that conflicts will happen and so it prevents them by acquiring locks on the data. This means that a transaction will block other concurrent transactions from accessing the same data until it completes. There are two types of pessimistic locks: Shared lock and Exclusive lock.

Shared Lock: A shared lock allows multiple transactions to read the same data concurrently but prevents them from updating it.

Exclusive Lock: An exclusive lock restricts access to data to a single transaction, preventing other transactions from reading or updating it.

Pessimistic locking is often used when the likelihood of conflicts is high.

When selecting rows in postgreSQL, you can use FOR UPDATE, FOR NO KEY UPDATE, FOR SHARE and FOR KEY SHARE locking modes to explicitly lock the selected rows. Each mode provides a different level of restriction in terms of which operations other concurrent transactions are allowed to do.

Here is an example of using SELECT FOR UPDATE, which creates an exclusive lock.

BEGIN;
SELECT * FROM products WHERE id = 1 FOR UPDATE;
-- Other transactions trying to read or update the same row will be blocked until this transaction completes
UPDATE products SET price = 10 WHERE id = 1;
COMMIT;

Please note that normal SELECT statements under an isolation level like READ COMMITTED will not be blocked by a SELECT FOR UPDATE or any other locking mode, as normal SELECT statements don't acquire any locks on the rows they read, so they don't cause any conflicts with other transactions.

Lock modes ordered by restrictiveness from highest to lowest:

SELECT .. FOR UPDATE (exclusive lock)
- Blocks concurrent transactions on the same rows from performing SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE, SELECT FOR KEY SHARE, UPDATE, and DELETE.
- This is the mode implicitly used for DELETE commands, and UPDATE commands on columns with unique indexes.
- Use when you want to prevent other transactions from modifying or acquiring any type of lock on the selected rows.
SELECT .. FOR NO KEY UPDATE (shared lock)
- Blocks concurrent transactions on the same rows from performing SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE, DELETE, and any UPDATE that changes key values (such as primary keys).
- This is the mode implicitly used for UPDATE commands that don't acquire a FOR UPDATE lock.
- Use when you want to lock selected rows against concurrent updates that change key values and from exclusive locks, while allowing other transactions to acquire SELECT FOR KEY SHARE locks.
SELECT .. FOR SHARE (shared lock)
- Blocks concurrent transactions on the same rows from performing SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, UPDATE, and DELETE.
- Use when you want to lock selected rows against all concurrent updates and from exclusive locks, while allowing other transactions to acquire SELECT FOR SHARE or SELECT FOR KEY SHARE locks.
SELECT .. FOR KEY SHARE (shared lock)
- Blocks concurrent transactions on the same rows from performing SELECT FOR UPDATE, DELETE, and any UPDATE that changes key values (such as primary keys).
- Use when you want to lock selected rows against concurrent updates that change key values and from exclusive locks, while allowing other transactions to acquire any type of shared lock.

Deadlocks

A deadlock happens when two or more transactions are waiting for each other to release locks. PostgreSQL periodically checks for deadlocks and handles them.

If a lock isn't released within the configurable parameter deadlock_timeout (default is 1 second), PostgreSQL will trigger the deadlock detection algorithm and if a deadlock was found, one of the involved transactions will be rolled back.

Optimistic Locking

Optimistic locking doesn't assume that conflicts will happen, so it allows all transactions to run concurrently. If a conflict is detected at commit time (two concurrent transactions have modified the same data), one transaction will proceed and the other will be rolled back and must retry.

Optimistic locking is often used when the likelihood of conflicts is low.

In PostgreSQL, you can implement optimistic locking using a versioning field in your tables. Here's an example:

CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    name TEXT,
    price NUMERIC,
    version INT DEFAULT 1
);

When updating a row, increment the version and include it in the WHERE clause:

UPDATE 
    products 
SET 
    price = 10.0,
    version = version + 1
WHERE
    id = 1 AND version = 1
RETURNING count(id);

If no rows are updated, it means another transaction has already updated the row and your transaction should retry.

Conclusion

I hope you found this summary helpful. Please let me know if you discovered any errors or have any suggestions on what to add or update, while still keeping it concise.

To-Do Lists for the Rescue

Mark Adel — Thu, 18 Jan 2024 19:13:02 +0000

In this article, I'm going to share my personal experience about the benefits of using to-do lists and share some tips that I developed over time which helped me use my to-do lists effectively.

I'm aware that there are countless resources over the internet on topics related to to-do lists, and time management in general. However, I believe that sharing personal experiences always adds new perspectives. I didn't do any research before or during writing this article to ensure that everything shared here is based solely on my experience.

Why should we use to-do lists?

My life before using to-do lists has been very messy. To-do lists helped me bring more order to my life, be more productive, overcome procrastination, start achieving my goals, and more.

Let's discuss the benefits of using to-do lists and why relying on our brains is not a good idea.

Our brains are unreliable

Limited memory

You can barely use your memory to remember today's tasks, and even for that it sometimes fails you and forgets things. How about if you want to keep track of short or long term tasks? This list can grow long, and relying on your memory becomes impractical.

By using to-do lists, there is no need to use your memory.

Overwhelm

Even if you manage to keep all the tasks in your memory, you will probably get overwhelmed by their amount, the size of each task, and the lack of clarity regarding their priorities or the order in which they should be executed. So you likely end up doing nothing.

By using to-do lists, you can organize the tasks in multiple lists, set their priorities and their order, and break down big tasks into smaller ones. Doing this eliminates the possibility of getting overwhelmed.

They motivate you to get things done

After filling out my daily to-do list, I find that I become motivated to start working on the tasks and getting them done. This results in less overall procrastination and wasted time.

They add structure to your day

Without using a to-do list, your day becomes subject to being very messy, since nothing stops you from doing anything at any time. So, you either end up wasting a lot of your time, like scrolling indefinitely in social media apps, playing video games, etc., or you find something to do that you think is important, just to not feel guilty for wasting your entire day.

A to-do list adds structure to your day. You are always able to answer the question, "What should I be doing right now?" All you need to do is consult your to-do list, which should have today's tasks ordered by their priority. While this doesn't stop you from wasting time, wasting time now becomes a conscious choice, rather than a result of an unplanned day, making you less likely to waste significant amounts of time.

They encourage you to set goals

I didn't have my short or long-term goals written anywhere before. Using to-do lists encouraged me to start documenting my goals and breaking them down into actions. I ensure that my daily to-do list includes at least one or two tasks that contribute to my goals.

They help you set priorities

By having all the tasks clearly written in front of you, it becomes more feasible to prioritize them.

Tip: Not all tasks that seem important are important. Always ask yourself whether these tasks, especially big ones, contribute to one of your current goals. If not, they are probably not due now, and there are more important ones that directly impact your goals.

How to structure to-do lists

There is no right, wrong, or even recommended way to structure your to-do lists. You can structure them in any way you like, as long as they are effective and achieve the benefits we discussed in the previous section.

I'm going to share two of the lists that I use, which I believe everyone can make use of, as these are the ones we use in our day-to-day lives: "Daily to-do list" and "Pool of to-dos".

Other lists that you may consider having, just to give you an idea: Goals for Q1 2024, Shopping list, Places to visit, Future tasks (tasks not immediately necessary but you don't want to forget), and more.

Let's explore the Daily to-do list and the Pool of to-dos.

1. Daily to-do list

We are all familiar with this one. This list contains tasks to be completed within a specific day. It should be prepared at the start of the day and updated as necessary throughout the day.

2. Pool of to-dos

This list is used for tasks that are ready to be executed, but can't all be squeezed into today's to-dos. It serves as a pool of tasks waiting for their turn to be moved to the Daily to-do list when the time is right.

Whenever a task of this nature comes up, drop it to your pool of to-dos.

Tips for using to-do lists effectively

Following are some tips that I developed over time for making the most out of the to-do lists. These tips specifically apply to the Daily to-do list and the Pool of to-dos.

Keep Pool of to-dos and Daily to-do list in the same place

These two lists are often used together, so it makes sense to keep them close. Ideally, you should be able to drag and drop tasks between them, as this is something you will do all the time.

Your to-do lists should be easily accessible

Ensure that the place where you keep your to-do lists is no more than two clicks away on any device that you use.

Keep tasks small

Ensure that each task takes no longer than two hours to complete unless there are specific exceptions.

Big tasks should be broken down into smaller, more manageable ones.

Having big tasks on your list can lead to feeling overwhelmed, which increases the likelihood to procrastinate. Additionally, keeping your tasks small allows you to work on multiple items each day.

Writing this article was a big task on my daily to-do list. It wasn't necessary to complete it in one or two days, so I kept it on my daily to-do list with a note "(recurring daily until completed)". I would then work on it for an hour or two each day, then mark it as completed for this day.

Prioritize your tasks

Order your tasks based on their priority. In the case of equally important tasks, I usually start with the more annoying ones. By doing this, you ensure that you will complete the important tasks each day.

Please note that it's important to prioritize tasks on your pool of to-dos too.

Be realistic

When filling your Daily to-do list, avoid adding more tasks than you can complete, considering your energy and available time for the day.

Start with yesterday's leftovers

If for some reason you didn't complete all of yesterday's tasks, consider starting with the uncompleted ones. This prevents uncompleted tasks from lingering on your daily to-do list for multiple days.

If an uncompleted task stays in your daily to-do list for more than two days, consider moving it back to the pool of to-dos, and only move it to your daily to-do list when you are ready to complete it.

Ensure that tasks on your pool of to-dos do not rot

Some tasks may stay in your pool of to-dos for many days or even weeks. To handle this problem, from time to time, I go over my pool of to-dos and try to include one or two of these tasks in my daily to-do list, regardless of their priority.

Don't sacrifice sleep

Don't stay up because you still have unfinished tasks. Instead, strive to finish all the tasks on your daily to-do list before your regular bedtime.

Set deadlines

I don't have the need to set deadlines for my daily to-do list tasks, but you may find it beneficial. It depends on your work style and the nature of the tasks at hand.

Keep it clean

I prefer to use correct language when writing the tasks, avoiding spelling or grammar mistakes. The only time I violate this is when I need to drop a task into my pool of to-dos while I'm in the middle of doing something else. I would fix the language later.

Include playing too

I like playing blitz chess (3 minutes time format). Chess used to consume a good portion of my free time. After I started using to-do lists, I decided to include chess in my daily tasks. I would add two or three tasks labeled "Play 2 chess games" to my daily to-do list. This not only acted as a motivation to complete the preceding tasks to get to each playing task, but it also helped limit the number of games I play. Previously, nothing prevented me from playing over 15 back-to-back games each time I started to play.

Conclusion

This concludes the end of this article. Please note that everything written is based on my own experience. What works for me might not necessarily work for you. I hope I have given you ideas on how to use your to-do lists effectively, or inspired you to start using them if you haven't already.

I may update this article as I learn new things, so keep an eye out for it!

A Practical Example of Using SQL Self Joins

Mark Adel — Wed, 03 Jan 2024 18:33:12 +0000

Problem

In this article, I am going to share a scenario that I recently faced at work and made use of SQL self joins.

Our company operates a food ordering app. We noticed that some customers add items to their basket, and for one or more hours, do not proceed to create an order. To address this, we decided to send these customers a push notification to encourage them to complete the ordering process. And that was my task. In this article, we will focus on how to retrieve these customers.

Solution

We have a relational table called customer_events where we store various events created by our app customers, such as: 'opened_app', 'added_items_to_basket', 'created_order', 'closed_app', and so on.

While Time-Series Databases could be a more suitable solution for this type of data, given our specific scenario and scale, we decided to stick with the traditional relational database.

Following is a sample of the customer_events table:

id	customer_id	event_type	event_timestamp
1	20	opened_app	2023-01-01 11:00:00
2	30	added_items_to_basket	2023-01-01 12:00:00
3	30	closed_app	2023-01-01 12:10:00
4	40	added_items_to_basket	2023-01-01 13:00:00
5	40	created_order	2023-01-01 13:30:00
6	50	added_items_to_basket	2023-01-01 15:00:00
7	60	added_items_to_basket	2023-01-01 16:30:00

We have events from 5 customers. Only the customer_id of 40 added items to their basket and then created an order. The others did not, so we want to retrieve these customers to send them a push notification, which are customer_id of 30, 50, and 60.

Before writing non-straightforward SQL queries, I like to use plain text to describe exactly what I want to achieve, in a way that would help me translate that into the query. Let's see what I would write for our case:

"I want to retrieve customer IDs that their event_type = 'added_items_to_basket' and their event_timestamp is one or more hours ago, excluding those for whom there exists a row with the same customer_id and event_name = 'created_order' and a greater event_timestamp"

Let's start with the easy part of the query, which is selecting all customers who added items to their basket at least one hour ago:

SELECT
    customer_id
FROM
    customer_events
WHERE
    event_type = 'added_items_to_basket'
    AND (EXTRACT(EPOCH FROM (NOW() - event_timestamp)) / 3600) >= 1;

Now, let's update the query to exclude those customers who created an order after they added items to the basket. We can do this using self joins:

SELECT
    a.customer_id
FROM
    customer_events a
LEFT JOIN
    customer_events b
    ON a.customer_id = b.customer_id
    AND b.event_type = 'created_order'
    AND b.event_timestamp > a.event_timestamp
WHERE
    a.event_type = 'added_items_to_basket'
    AND (EXTRACT(EPOCH FROM (NOW() - a.event_timestamp)) / 3600) >= 1
    AND b.customer_id IS NULL;

To the untrained eye, this query might look intimidating, but it's actually quite simple. Let's break it down.

We performed a LEFT JOIN on the customer_events table (a) with itself (b). This retrieves all rows from the left table (a) along with their matching rows from the right table (b) based on the specified join conditions. In the right side, all columns will be NULL if there are no subsequent 'created_order' event for the customer on the left side, and will have values otherwise. We use the b.customer_id IS NULL condition to only get customers where there are no subsequent 'created_order' event.

Query Output:

customer_id
30
50
60

If you are not sure what the output of the self join looks like before filtering using b.customer_id IS NULL, you may run this example I prepared on db-fiddle.

It is worth noting that this is a simplified version of the actual query. There are other cases that we need to handle, such as ensuring that the order belongs to the specified basket, or only taking the last event into account. However, this is beyond the scope of this article.

Now, I want you to go back and read the plain text we wrote for this query, and then check the query again. You will find that it does exactly what we described.

Another Approach

There is an alternative way we can write this query to achieve the same result, using sub-queries:

SELECT
    a.customer_id
FROM
    customer_events a
WHERE
    a.event_type = 'added_items_to_basket'
    AND (EXTRACT(EPOCH FROM (NOW() - a.event_timestamp)) / 3600) >= 1
    AND NOT EXISTS (
        SELECT 1
        FROM customer_events b
        WHERE a.customer_id = b.customer_id
        AND b.event_type = 'created_order'
        AND b.event_timestamp > a.event_timestamp
    );

Which approach to use? We have two factors to consider: readability and performance.

In terms of readability, it is a personal preference. For me, I find both equally readable.

In terms of performance, it depends on many factors. It is always advisable to check the execution plans of your queries.

Conclusion

Whenever you find yourself in a situation where you want to exclude certain rows from your result based on conditions that involve other rows in the same table, SQL self joins can come in handy.

I hope you found this article helpful. Please let me know if you have any feedback. Happy querying!

DEV Community: Mark Adel

A Philosophy of Software Design Summary

Chapter 1, 2: Introduction and The Nature of Complexity

Chapter 3: Working Code Isn't Enough

Chapter 4: Modules Should Be Deep

Chapter 5: Information Hiding (and Leakage)

Chapter 6: General-Purpose Modules are Deeper

Chapter 7: Different Layer, Different Abstraction

Chapter 8: Pull Complexity Downwards

Chapter 9: Better Together Or Better Apart?

Chapter 10: Define Errors Out Of Existence

Chapter 11: Design it Twice

Chapter 12: Why Write Comments? The Four Excuses

Chapter 13: Comments Should Describe Things that Aren’t Obvious from the Code

Chapter 14: Choosing Names

Chapter 15: Write The Comments First (Use Comments As Part Of The Design Process)

Chapter 16: Modifying Existing Code

Chapter 17: Consistency

Chapter 18: Code Should be Obvious

Chapter 19: Software Trends

Chapter 20: Designing for Performance

Chapter 21: Decide What Matters

Chapter 22: Conclusion

Modern Software Engineering Chapter 4–8 Summary

Chapter 4 (Working Iteratively)

Chapter 5 (Feedback)

Chapter 6 (Incrementalism)

Chapter 7 (Empiricism)

Chapter 8 (Being Experimental)

Make It Work, Then Make It Better

Make It Work

Make It Better

Conclusion

Fixing VS Code Go to Definition Issue on Linux Mint

Solution

Logging Done Right

1. All log messages should start with the class and function name

2. Add logs in all exception handling blocks

3. Include context if useful

4. Add logs for auditing

5. Don't include large log messages if not necessary

6. Don't log sensitive data

7. Use the correct log level

8. Timestamp your logs

9. Avoid logging in loops or recursive functions if not necessary

10. Log the start and end of long-running operations

11. Ensure log messages are concise and clear

Conclusion

References and Further Reading

Disaster Recovery: Better Safe Than Sorry

Conclusion

Diagnose Before You Prescribe

Backend Code Review Checklist

Code Style

Code Maintainability

Requirements

API Design

Documentation and Comments

Error Handling

Security

Dependencies

Logging

Testing

Performance

Version Control

Spelling

Conclusion

Useful Global Git Configurations

1. git config --global init.defaultBranch main

2. git config --global help.autoCorrect prompt

3. git config --global branch.sort -committerdate

4. git config --global fetch.prune true

5. git config --global log.date iso

6. git config --global push.autoSetupRemote true

Conclusion

PostgreSQL Isolation Levels and Locking Summary

Read Phenomena

Isolation Levels

Read Uncommitted

Read Committed

1. `git config --global init.defaultBranch main`

2. `git config --global help.autoCorrect prompt`

3. `git config --global branch.sort -committerdate`

4. `git config --global fetch.prune true`

5. `git config --global log.date iso`

6. `git config --global push.autoSetupRemote true`

`Read Uncommitted`

`Read Committed`

`Repeatable Read`

`Serializable`