DEV Community: konrad_126

Keep calm and git bisect

konrad_126 — Sun, 20 Dec 2020 14:17:55 +0000

Your app broke, and you have no idea why? Even worse, you don't even know what commit is causing it to fail? Don't worry - it happens.

It is important to keep calm and track that first commit that is causing the failure, and git can help you out. Let's see how...

Finding the (first) bad apple

You start with what you know for sure - the last (present) commit is bad:

Then you checkout to some (potentially distant) commit in the past where you know things worked (you check it). Let's call it the good commit:

You can be sure that the first bad commit is somewhere between that good commit and the present bad commit:

To trace it, you can sequentially go through all those in-between commits and check your app at every step.

This process will work fine if there are a few commits to check, but what if there are tens or hundreds?

Because git's commit-graph, in this context, is a sorted list, we can speed up the process by using an algorithm called binary search .

Binary search to the rescue!

You start the same way as before - finding some commit in the (distant) past where things were nice and green:

Now, instead of sequentially checking all the in-between commits, you pick the commit in the middle of your list and check your app's behavior at that point:

If everything works you can consider that commit as a good commit:

You can also be sure that all the commits before it are also good so you don't need to check them individually:

If, on the other hand, the commit in the middle was bad (your app was broken):

Then you'd know for sure that all the commits after it are also bad:

Either way, you end up with a new list of commits to check that's half in size (with a new god/bad commit pair).

You repeat this process until you pinpoint the exact commit where things went sour.

Using binary search significantly cuts the time needed to trace the first bad commit, but all this manual checking out to commits still takes time and it's not much fun. Luckily, git has a command called ** bisect ** that will do that for you. All you'll need to do is to decide whether a certain commit is good or bad and git will do the rest.

Bisect

To start the bisecting process you type:

Then you tell git what commit is bad. If you omit the checksum git will assume it's the last commit on your current branch.

After you tell git what is the "good" commit:

it will display how many steps are left and immediately checkout to that first (middle) commit. Now you can check your app's behavior and tell git if the commit is good or bad:

Once you do, git moves (checks-out) to the next commit and you repeat the process. After a couple of steps (depending on how big your commit list was), you'll reach the final step and git will inform you what the first bad commit is:

Full Automation

Manually checking those commits to decide if they are good or bad is sometimes necessary, maybe it's some legacy app and there's not a lot of tests so you need to manually check your app. If you are lucky and can write a script to check it for you then becomes fully automated.

The script needs to be outside of source control (maybe a folder "up") and return exit code 0 if the commit is considered good or 1 if it's considered bad.

You start the bisect as before, by telling git which commit is good and which is bad.

But now you can tell git to run the bisect process and use your script to decide whether some commit is good or bad:

Now you know what commit was the one to introduce the failure (bug), hopefully, you'll be able to figure out the why part of the question.

Hexagonal Architecture Demystified

konrad_126 — Sun, 14 Jun 2020 08:38:55 +0000

There’s no shortage of lectures and blogposts on “Hexagonal Architecture” these days, but many of them give a much broader view of Hexagonal Architecture than it was originally presented by its author Alistar Cockburn. This can make you feel overwhelmed, so let’s re-visit Alistar's original idea and remove some of the confusion.

We’ll start the same way Alistar did, by looking at traditional Layered Architecture first.

Layered Architecture

To help ourselves deal with the complexity of software systems we can divide them into separate layers (based on their levels of abstraction). There’s no limit to the number of layers but authors usually suggest three or four. Here we’re going to use the four-layer model from the blue DDD book:

User Interface Layer — Displays some data to end-users and where end-users interact with the system
Application Layer — Orchestrates Domain objects to perform tasks required by the end-users
Domain Layer — Contains all business logic, the Entities, Events and any other object type that contains Business Logic
Infrastructure Layer — Technical capabilities that support the layers above, e.g. persistence or messaging

Communication between the layers only goes downwards. In a strict approach, a layer can talk only to a layer directly below it, while in the relaxed approach a layer can talk to all layers below it in the layer stack.

Traditionally, we also refer to the UI layer as the Front End of our application and the Infrastructure layer as the Back End. The “middle” part I chose to call the Core, but you could also call it the Domain, Business logic, or maybe even the Heart of Software if you wish to be more poetic.

A different view of a system…

In Alistar’s eyes, there is no big difference between the UI (our web client) and the Database. A system is made of only two distinct parts: the inside and the outside. The inside is our Core and the outside is where the UI and Infrastructure live.

In this new view of a system, the Database and the Web Client are no longer back end and front end. They are both the same - outside.

In order for Core and the outside world to be able to talk to each other, we need a way of transforming data between different formats they use. For example, our web client speaks HTTP while our application speaks PHP. That is why there is another hexagon wrapping our Core.

In this wrapper, the HTTP requests are transformed into some PHP data structure that our Core can understand, and vice versa. The same goes for our database. We need a way of transforming our PHP data structure (probably objects) into a format of our database (probably relational database). We can call this wrapper the transformer wrapper and this is where Ports and Adapters come into the story.

Ports and Adapters

Ports live in your core and they define how to talk to the Core. To do so you need to plug an adapter into them that will transform the input from the outside world format to one accepted by the port.

Going back to our application. In the Core, we have some Use Case. Maybe it’s a command if you are using the Command Query Separation approach or it is just a Service class with a public method. In any case, our web client speaks HTTP and our Core speaks PHP so we need to transform HTTP into PHP and vice versa. So we write our adapter that does just that. This is called a Controller.

Now we want to provide the outside world with another way of talking to our Core. Maybe with a CLI. All we have to do is to write another adapter that will take the input from the CLI and transform it into that same PHP format our Core can understand.

Notice how we didn’t need to touch the code in the Core to support this new method of talking to our system. The Core is blissfully ignorant of the many different ways of talking to our system from the outside world (it only cares about ports).

The system can be described as a Core surrounded with interfaces to the outside world. Alistar chose to visualize the Core as a Hexagon because it was easier to draw than the pentagon, but also because it gives us a nice visual analogy where one edge of the Hexagon represents one reason of talking to the outside world. There is nothing magical about the number six — you can have a shape with n-edges.

As we already mentioned, the UI and the Infrastructure are no longer back end and front end but simply outside. So, there is a certain symmetry in this view of the system. However, there is a certain asymmetry to it as well. What that means is that we can divide our ports into two groups: driving ports and driven ports.

Ports are divided based on how they communicate with the Core. When it comes to driving ports, they are the ones initiating the communication with the Core (driving the behavior of our application). In the case of driven ports it is the Core that initiates the communication.

And this is Hexagonal/Ports And Adapters architecture as presented originally by Alistar. He gave no instructions on how you should structure the code in your Core. Many people tend to use some layering (onion) and mix other architectures with Hexagonal. if you wish to explore more on that I recommend this article from Mattias Noback and this one from Herberto Graca.

Why?

Using Ports and Adapters will add some complexity to your architecture and design (as any design or architectural pattern will), so let’s review what some of the main benefits of this architecture are.

Delay technical decisions

There is a paradox in how we develop software. At the beginning of a project, when your project (domain) knowledge is at a minimum, we make big technical decisions. Our domain knowledge will only grow over time so wouldn’t it be nice if we could delay those technical decisions to some point in the future when we’ll be able to make a more informed decision?

With Ports and Adapters, we can do just that. Say you’re starting a new project and you are not sure what kind of database you should use. You can write your repository adapters in plain PHP that will serialize your entities and save them to a text file. Sure, this solution is not production-ready, but it will allow you to model your domain and to test it, and once you are much more confident about the domain you can pick your database and write real adapters for it.

Swap technologies

What about changing your mind? Maybe you realized your choice was wrong, or there is simply a new better technology you want to use.

Switching between technologies is simply a matter of writing new adapters for those technologies. This, of course, can be hard work but it consists of writing new code, which is always cleaner than modifying the existing one (and why the Open-Closed Principle is such a good thing).

Testability

When it comes to testing, Ports and Adapters architecture enables us to test our application in isolation from external dependencies.

On the driving side, we can swap our real adapters with testing adapters (our test suite), so when testing business rules in the Core, we don’t need to talk to it through the UI, we can plug in our test suit directly to the Core.

On the driven side we can swap the driven adapters with testing adapter implementations:

Don’t get me wrong. You should still write some integration tests to check if your real-infrastructure adapters work, but you don’t need to test that integration every time you’re testing some business rule in your Core.

Focus on the core

We mentioned earlier that our core is blissfully unaware of the outside world — the adapters. The only thing our core cares about is the ports that it provides to the outside world. This enables you to not be influenced by technology decisions while developing your Core, so it can be truly domain-driven instead of technology (framework) driven. Now, having you Core entirely free of any technology influence leakage is maybe a bit idealistic, but still, this architecture gives you a high level of decoupling between your domain and business logic and the technologies you use.

When

So when should you go for using Ports and Adapters architecture? Well, as always, it depends. It does add some complexity to your design so you need to decide does it pay off. If the benefits it brings are important to you then I’d say go for it.

Fun Driven Development with PHPSpec

konrad_126 — Fri, 05 Jun 2020 17:39:15 +0000

The first time I saw the abbreviation BDD I thought it was a miss-print of TDD (Test Driven Development), but it’s not. It stands for Behaviour Driven Development and it is a technique derived from TDD. The difference between the two is a bit nuanced and best felt when you start using some BDD tools. One such tool is PHPSpec. It will help you write better code and have fun in the process. Sounds good? Let’s give it a try!

There is a dog

Dogs are fun so let's build a Dog game:

In PHPSpec you start by creating the specification first. It will describe the Dog class we are going to build:

PHPSpec generates the specification file for us:

<?php

namespace spec\App;

use App\Dog;
use PhpSpec\ObjectBehavior;
use Prophecy\Argument;

class DogSpec extends ObjectBehavior
{
    function it_is_initializable()
    {
        $this->shouldHaveType(Dog::class);
    }
}

It doesn't say (describe) much at the moment - just that the class Dog should be initializable.

Let's run PHPSPec to see what happens.

Whoa, that's a lot of pink! What's up with that? Well, this is how PHPSPec indicates that the code is broken (doesn't even run), which is different than it running but returning wrong results. It is also asking us to fix it by creating the missing Dog class for us. So we comply and we get this:

<?php

namespace App;

class Dog
{
}

After creating the Dog class, it runs again and now we have our first spec passing giving us that green light all devs love so much.

Dogs make sounds

As all dog owners will tell you, dogs bark, so let's add this behavior to our doggo:

We describe the desired behaviour in the spec class:

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
    // ...

    function it_should_make_a_sound()
    {
        $this->makeSound()->shouldBe('Vuf Vuf');
    }

}

I know. This looks a bit wired. We are calling the makeSound method but there is no such method in our spec class. And there won't be. The $this keyword in a spec class doesn't refer to the spec class itself, but to the class the spec is describing. In this case, DogSpec described the Dog class, so PHPSpec will try to call the makeSound method on the Dog class. (There is always one spec class per class).

Another wired thing is the shouldBe method chained to the makeSound method. This is what is called a matcher in PHPSpec and it's equivalent to PHPUnit's assertions (and there's a bunch of them). In this case, it will take the output of the makeSound method and check if it matches Vuf Vuf.

Now that we've straightened that out, we can run PHPSpec again:

Since we don't have the makeSound() method on our Dog class yet, there's that purple color again and there's PHPSpec again being nice and helpful offering to create the method for us.

After creating the method it runs again and now we finally have some red color meaning the code works but not as expected.

Time to write some code and get us back to green!

<?php

# src/App

class Dog
{
    public function makeSound(): string
    {
        return 'Vuf Vuf';
    }
}

Stubs

Our object often time collaborate with other objects by asking them questions (and doing something with the answer). If we go back to our pet game, dogs are usually very happy to great people:

To describe this in our specification we'll need a stub:

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
    // ...

    function it_can_greet_a_person(Person $person)
    {
        $person->name()->willReturn('Mike');
        $this->greet($person)->shouldBe('Hello Mike, Vuf Vuf');
    }
}

Looking at this, you may be wondering where is some createTestDouble() method call? In PHPSpec you just inject classes/objects you need and PHPSpec will create test doubles out of them (using prophecy framework) so you can immediately configure them. Here, we are stating the person's class name() method will return Mike when called.

By now you can assume when we run PHPSpec it will offer to create the actual Person class for us, right?

But wait. It says "Would you like me to generate an interface". Why not a class? Hm, now that I think about it, dogs also like to greet other dogs not just humans, so maybe there is an abstraction here that we missed. Let's introduce an interface called Named and make our Dog class is dependent on an interface instead of a concrete class.

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
    // ...

    function it_can_greet_a_person(Named $named)
    {
        $named->name()->willReturn('Mike');
        $this->greet($named)->shouldBe('Hello Mike, Vuf Vuf');
    }
}

Now our Dog can greet any class that implements the Named interface. This design will make it easy to expand the list of earthlings our doggos can greet - just implement the Named interface. Thnx PHPSpec!

We can let PHPSpec make that interface and the greet method for us now

And now we can implement our greet method to get us back to green again:

<?php

# src/App

class Dog
{
    public function makeSound()
    {
        return 'Vuf Vuf';
    }

    public function greet(Named $named)
    {
        return 'Hello ' . $named->name() . 'Vuf Vuf';
    }
}

Mocks and Spies

Let's say for business reason, every time a Dog greets someone we want to our application to log that information.

In this case, our object is going to issue commands to other objects, so we need a mock or spy. First, we are going to state that Dog will be constructed with a Logger:

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
    function let(Logger $logger)
    {
        $this->beConstructedWith($logger);
    }
}

Now we can describe the interaction with a Logger:

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
     /// ...
    function it_logs_the_greeting(Named $named, Logger $logger)
    {
        $named->name()->willReturn('Marry');
        $logger->log('Hello Marry, Vuf Vuf')->shouldBeCalled();
        $this->greet($named);
    }
}

We are setting an expectation that the log method will be called with Hello Mike, Vuf Vuf once we execute our code. This is a mock. If we wish to use a spy we would describe it as:

<?php

# spec/App

class DogSpec extends ObjectBehavior
{
     /// ...
    function it_logs_the_greeting(Named $named, Logger $logger)
    {
        $named->name()->willReturn('Mike');
        $this->greet($named);
        $logger->log('Hello Mike, Vuf Vuf')->shouldHaveBeCalled();
    }
}

As always, PHPSpec will help us by creating some methods or us:

Then it's up to us to add some code to satisfy the spec:

<?php

# src/App

class Dog
{

    // ...

    public function greet(Named $named)
    {
        $greeting = 'Hello ' . $named->name() . ', Vuf Vuf';
        $this->logger->log($greeting);
        return $greeting;
    }
}

As you see, in PHPSpec we describe our object behaviors using a very descriptive syntax. There is always one spec per object/class which is why PHPSpec can only be used for testing at the unit layer/level. If you want to have some higher-level test you'll need to use some other tool. Maybe another BDD tool as Behat (don't get me started on how cool Behat is)? Another thing you probably don't want to do is introduce PHPSpec in a legacy codebase (with bad design). It will be painful and authors of PHPSpec never intended it to be used that way. But if you are starting a greenfield project, why not give PHPSpec a try. Hope you'll find is as fun as I do.

Pointers and tips: dispelling the magic of git merge

konrad_126 — Sun, 31 May 2020 09:52:03 +0000

Creating branches in git is blazingly fast and having a bunch of them is pretty cheap. This means we get to merge them quite often. But how is a branch represented internally and what does it mean to merge them? Understanding how this works internally will help you understand why merge conflicts occur. Let’s dispel the magic.

Branches as pointers

If we visualize git’s commit history, each commit is represented as a node on a graph. Each commit has a connection to its parent; we can only travel from a commit to its parent(s) when traversing the graph:

We know in reality git uses those random-looking strings as commit identifiers (checksum headers) so let’s use them too:

It’s very intuitive to think of a branch as a list of commits, but a branch in git is just a pointer (reference) to a commit:

The commit that the branch references is the last commit on that branch, so we also call it the tip of the branch.

We know that when we use git log command, git shows a list of all commits on the branch, not just a single commit, so how does this work?

Well, if and when git needs the full list of commits that a branch contains (in this example master), it will generate it by traversing the commit-graph starting from the tip of the branch:

You can still think of branches as lists of commits, but git doesn’t need to keep this full list stored somewhere all the time. It only needs to keep track of the tips of the branches and simply generate the full list when it’s needed

Everything is in the commit

We can think of commits as points in time in the history of our project. Every one of these points represents the changes made to the project at that point. In git, every commit also includes the full state of your project at the point of the commit.

This means when you checkout to a specific branch git doesn’t need to traverse its full list of commits (applying changes from each commit) to recreate the state of your project (to match that branch). It only needs to look at the tip of the branch.

The reality is a bit more complex but you can think of it as every commit containing a list of all the files in your project with the version identifier of the file. In this simple example we have only three files in our project and have committed the initial version of them in our first commit:

If we change the kernel.php file and commit those changes, the next commit will still contain the same list of files, but the version identifier for kernel.php will be updated.

Git uses these version identifiers to quickly search its data storage for the object containing the content of that exact version of the file. Different version identifiers for the same file mean the content of the file has changed.

Now that we know how git branches are represented at a low-level, let’s see what it means to merge them.

Merge

Going back to our example, suppose our shiny new feature is done and we want to merge it to the master branch. In other words, we want the changes we did on the feature branch to be present on the master branch as well.

If we traverse the commit-graph starting from the tip of the master branch, we can see that those two commits from the feature branch are not accessible from that tip. This, in turn, means the changes we did on those commits are also not present on the master branch.

The result we want to achieve by the merge is to make those commits accessible from the tip of the master branch.

The first step in the merge is to find the merge base commit. This is the commit after which our branches diverged. Git finds it by going to the tips of both feature and master branches and traversing the commit-graph until it finds the first commit that is accessible from the tips of both branches:

Once it has located the merge base, git will start building the new merge commit. This merge commit, once it’s done, will have an interesting property — it will have two parents.

The first parent will be the current tip of the master branch and the second the current tip of the feature branch. Because of this property, the new merge commit will have access to all the commits from the master branch but also to those two new feature branch commits as well.

To construct that new merge commit git needs to decide what versions of our files go into it. To do so, git will compare information from the merge base and the tips of master and feature branches:

The first file in the commit’s list is app.php. Git compares the file version identifiers in the merge base commit (539) with the ones in the tips of master and feature. The version identifier on the master branch (df5) is different than in the merge base (24e) while the tip of the feature branch has the same identifier (24e) as the merge base. This means we have only changed the app.php on the master branch so this is the version git will put in the new merge commit.

The next file in the list is kernel.php. The situation is a bit simpler now since we didn’t change this file on either of our branches. Git knows this because the version identifier is the same in all 3 places (ea3) so git will take the version from the base commit and put it into the merge commit.

Finally, when looking at the last file on the list (README.md), git sees we changed it on the feature branch only so it’s again smart enough to realize that it can take that version (changes) and put it into the new merge commit.

Once the merge commit is constructed the only step left is to update the master branch pointer to point to the new merge commit, which is now the new tip of the master branch.

From this new tip, those commits made on the feature branch are now accessible so the changes we did on those commits are also present on the master branch.

What about conflicts?

Git can be smart enough to do the merge automatically as long as we didn’t change the same file on both branches. If that is the case, we have a conflict and git needs our help.

Consider the scenario from before with a minor difference. This time when git is trying to construct the new merge commit it finds we have changed the README.md file on both of our branches.

Git cannot read our minds and know which of these file versions (changes) is more important to us, so it will stop the automatic merge process and ask for help.

The message git gives us is to fix the conflict in the README.md file and then manually create the merge commit by using the commit command. So, let’s fix that conflict.

If you open the README.md file you will see git has added textual markers to it to indicate the conflict.

The content between the <<<<<<< HEAD and =======is how this line of the README.md file looks on the master branch while the content between the ======= and >>>>>>> feature is how it looks on the feature branch. To resolve the conflict you simply need to make the file look as you want it. In this case, we want the Hello Mars text so we would delete the markers and the Hello Earth line:

Now we need to type git add README.md to let git know we have resolved the conflict and git commit to actually make the new merge commit. Git will give it the automatic Merge branch ‘feature’ commit message but you can also change it to whatever you want.

People usually use a GUI tool, such as SourceTree or the one built into your IDE, to resolve conflicts because manually inspecting the files and making sense of those textual markers git gives can become cumbersome.

Fast Forward

Because branches in git are just pointers to commits, merging can become quite a trivial operation at times. Consider this example:

We have created a feature branch and made some commits on it, but in the meanwhile, we didn’t do any changes (commits) on the master branch. We can say that the tip of the feature branch is ahead of the tip of the master branch (by two commits). This becomes even more clear if we slightly modify our visualization:

As we showed, to merge the feature branch into master means to make those two commits from the feature branch accessible from the tip of the master branch. To do so git can simply move the tip of the master branch forward:

The only caveat here is that git will not create that special merge commit, so by looking at the history log of the master branch, you won’t be able to tell it was merged with the feature branch at some point. If you do need this information, you can pass the — no-ff flag to the merge command and it will always create the merge commit.

To answer our questions from the beginning, branches in git are just pointers to commits (tips), and merging branches is all about making commits accessible from the tips of branches. I hope with this understanding of the way that git works you see through the magic. Now you will be better suited to tackle any problems you have with branching. So merge often, have fewer conflicts, and live long and prosper!

It’s contextual

konrad_126 — Thu, 28 May 2020 12:41:55 +0000

“There is nothing either good or bad, but thinking makes it so”

A timeless comment on the human condition by Shakespeare’s Hamlet. It came to my mind as I was attending GOTO Online Bookclub featuring Kevlin Hanney.

The topic of the day was Udi Dahan’s Beware the share essay from the 97 things every programmer should know book. In it, Udi shares a story of his first job. Coming fresh from college, eager to prove himself, he looked for every opportunity to improve the codebase by pulling out shared code into libraries, only to find out during code review, that other developers were not too happy about it. On the contrary — reuse was frowned upon!

Kevlin opened the session by pointing out how Udi’s essay stands in contrast to another essay from the same book — Don’t repeat yourself by Steve Smith, which is all about the dangers of duplicate code (logic).
So who is right, Udi or Steve? They both are.

The duality of life

This kind of duality is not unique to software development. We come across it often in life, but it can still confuse or even scare people. We tend to seek easy, one-size-fits-all, solutions and if both Udi and Steve are right then how do we choose whose advice to follow?

Later in his essay, Udi states he came to realize he missed the critical part when blindly applying his eliminate code duplication rule — context. It is the context that matters and creating those shared libraries also came at a cost (increasing the dependencies between code that is using those libraries).

Kevlin continued on this, generalizing it to all software design patterns, principles, guidelines… Their usability and benefits are dependent on the particular context you are applying them in. And “context is a function of time”, so you need to be ready to re-asses your decisions as time goes by. Decisions that were perfectly reasonable at the time you made them, can become wrong with the passage of time (change of context).

What to do?

Hamlet is right but accepting this fact doesn’t mean we should be paralyzed by our lack of total understanding. We should act to the best of abilities given the current context (knowledge). But we should also understand how highly contextual all our decisions are and be ready to re-asses them as the context changes. Design patterns, “rules” and guidelines are there to help us but we should not lose sight of the context in which they are to be applied.

Visualizing technical debt with Quality Views

konrad_126 — Fri, 15 May 2020 13:25:00 +0000

In his The Iceberg Secret, Revealed blog post, Joel Spolsky compares software with icebergs:

You know how an iceberg is 90% underwater? Well, most software is like that too — there’s a pretty user interface that takes about 10% of the work, and then 90% of the programming work is under the covers

Later in his text, he lets us in on a little secret:

People who aren’t programmers do not understand this.

And what's even harder for people who are not programmers to understand is how bad the "underwater" part can be (especially if the visible part looks fine). It can be hard to grasp, it can have fixes that are software development versions of duck taping a bike tire, it can be poorly or non documented...

You know what I'm talking about. Technical Debt. Just hearing the word makes your spine shiver. And communicating this to non-programmers can be very hard because

for a component that appears to be working, a non-programmer will assume the technical debt is minimal, or nonexistent.

During his Dealing with legacy code talk at the Software Craftsmanship meetup in Berlin, Michal Feathers briefly mentioned a technique of displaying technical depth visually. I didn't catch the name of the technique so I got him to write me a not after the talk:

It took some deciphering but I managed to get to the source. It was Colin Breck, an engineer/team lead at Tesla.

Inspired by a few different sources (including Michael Feathers talk on Conway’s Law) he developed a

way to represent complex systems, or systems that are not customer-facing—like infrastructure platforms—to people who are non-technical, or not intimately familiar with the software.

The main information this representation conveys is the internal quality of the system components. Hence the name, Quality Views.

The idea is rather simple. First, you represent your system with an architecture graph.

The size of the elements represents the "relative size of the servers in terms of memory, cores, and IO capacity". For example, legacy load balancer servers are smaller than the "new" ones. Dashed lines represent system boundaries and the arrows show the flow of data (the thicker the arrow the bigger the flow). So, a nice graph rich in information but nothing new. I'm sure you've drawn a graph or two similar to this one.

The next thing that Colin does is he adds "information related to software quality and how the system is evolving" to this graph.

The way he does that is rather simple. The quality is color-coded. So, in this case, the closer to red the worse the quality.

To evaluate the quality of each component Colin developed a set of criteria, with each criterion contributing to the overall quality grade of a component. His criteria are:

Code
Tests
Deployment
Monitoring
Alerting
Security
High-Availability
Scalability
Risks

And for each, there is a set of questions who serve as a checklist for each grade. Now, this is a set of criteria he developed for his use case, you can (and probably should) develop your own set of criteria. Also, you can introduce weights to these criteria because not all of them may be of the same importance to you.

Once you have your criteria set, you get a nice visual representation of Technical Debt that everyone, including non-technical people, can understand. Now, when planing a new feature, your manager can know how risky it is given the quality of components it requires work on. Also, this graph highlights the things you are not working on. And the risk of having a lot of (or relying on) low-quality components becomes harder to ignore since it's constantly present in this visual display.

Another thing, that can be a side effect of using Quality Views, is that it can also motivate your team. Internal code/infrastructure improvements often don't have a direct tangible effect on the application (the visible part of the iceberg) and using a quality view will give your developers (as well as PMs) just that - a tangible visual representation of the improvements done.

If you wish to explore Quality View more deeply I suggest you watch the presentation Using Quality Views to Tackle Tech Debt @Tesla Colin gave at QCon London, as well as reading his two blog posts on the subject: Using Quality Views to Communicate Software Quality and Evolution and Reflections on Using Quality Views.

* images are taken from https://www.infoq.com/presentations/quality-views-technical-debt/

Get, set... STOP!

konrad_126 — Tue, 12 May 2020 14:56:36 +0000

What if I told you Morpheus never said: “What if I told you?” I know you saw that meme a thousand times but have you actually heard him say it in the movie? You didn’t, because it didn’t happen. Darth Vader also never said “Luke, I am your father” and Captain Kirk never said, “Beam me up, Scotty”. People keep repeating these “quotes” so we assume they are true.

Cargo Cult

This phenomenon is not limited to spoken word — it afflicts human behaviors also. Since software is developed by humans (for now), we come across it in software development. It’s called cargo cult programming and Wikipedia defines it as:

“Ritual inclusion of code or program structures that serve no real purpose.”

One practice that is widespread among developers is the creation of getters and setters. They always come together and we create them as soon as the entity is created. This is so deeply engrained in our workflow that nowadays IDEs offer to automatically generate these getters and setters for us.

But is there a reason for doing this or is it just a ritual we have accepted without stopping to wonder why?

Set the lights to off

Let’s look at the setters first. By setters I mean methods like this:

public function setName(string $name)
{
  $this->name = $name ;
}

This method clearly changes the name of the object in question. In the real world, we call this renaming. I have never heard someone say that a person has “set its name to another name”. Even if you ask the developer who wrote the method chances are they will tell you that it renames the object. So, here’s an interesting proposition for you: if renaming is what the method does, call it rename.

public function rename(string $name)
{
  $this->name = $name ;
}

Now our method name describes precisely what our method does and we used a domain-specific word for it. The (domain) concept of renaming a user is reflected in our code. This gives developers and non-developers a common (ubiquitous) language when discussing the project.

Let’s look at another example. Say we have a Subscription class with the following setter method:

public function setActive(bool $active): void
{
   $this->active = $active;
}

As with the previous example, this method doesn’t clearly reflect the domain concept which is activating and deactivating a subscription. So we split it into:

public function activate(): void
{
     $this->active = true;
}

public function deactivate(): void
{
    $this->active = false;
}

Again we have expressed these concepts explicitly in our code; furthermore, there is another benefit gained by this refactoring. Having a setter method allowed the outside world to set the internal state of the Subscription object, whereas now, the Subscription object is responsible for setting its own internal state.

Anemic vs. Rich

By having myriad setter methods on your entity, you end up with what’s called the anemic domain model.

<?php 
class User {
    public function setName(string $name): void
    //...
    public function setBirthDate(DateTime $date): void
    //...
    public function setAddress(Address $name): void
    //...
}

Your User entity is more of a data structure than an object. Data structures expose their data and have little or no behavior while objects protect their data and expose behaviors.

To have a richer domain model, we refactor this to:

class User {
    public function __construct(string $name, DateTime $birthDate, Address $address)
      {
        $this->name = $name;
        $this->date = $birthDate;
        $this->address = $address;
      }

    public function rename(string $name): void
    //...
    public function updateAddress(Address $address): void
    //...
}

Now our User class is protecting its internal state (invariants) and exposing only behaviors to the outside world.

“Get me your name”

Now, to the getters. From my experience, this is harder for people to get (pun intended), and, to be clear, by getters I mean methods like this:

public function getName() : Name 
{
   return $this->name;
}

This method returns the name of the object in question. Many developers would say this is clear from its name. The get + Name tells us it returns the Name. Actually, the “get” prefix guarantees nothing. In actuality, it is the return type of the method that guarantees this method returns a Name.

If you ditch the get prefix in the method name then you lose no information in the process:

public function name() : Name 
{
   return $this->name;
}

If you can remove a word without losing any information, then that word is just noise.

But this is how we talk, right? Not really. Recall from memory a situation where you need to reveal some personal information about yourself such as being in a bank. When the bank clerks ask you for your address, do they say “Get me your address” or “Get me your name”? They don’t.

In fact, when we use the word get in real life it usually isn’t a question but an imperative that has some implied side effect. Say you want a beer so you tell your friend “Hey, get me a beer from the fridge”. The side effect of him getting you a beer is that the freezer will have one less beer afterward. This will happen every time you call him to get you a beer. I’ll admit, it would be nice if beer fridges would follow our programming conventions of “getters” not changing the internal state of the object, but they don’t.

Naming things in software development is hard and we want to be as precise as possible when doing it. What if I told you get and set are among the most Ambiguous words in the English language. Check google and see how long the list of their possible meanings is.

Maybe they aren’t the best candidates when you’re aiming for precision.

Makefiles for automation and better dev-UI

konrad_126 — Sun, 10 May 2020 13:48:50 +0000

As our projects grow in complexity, the list of tools they require also grows, and remembering how to use all of them (with their different syntaxes) can become cumbersome. On top of that, some tasks require multiple steps and tools to be run sequentially. We can create our own aliases and scripts for multi-step tasks, but the downside is remembering all those aliases and scripts.

Wouldn't it be nice if we could have a single CLI entry point to our project that would give us a way to list and run all the tools or tasks it contains? Something consistent across all projects, no matter the technology? Something like this?

Well, turns out we can have this with the help of make. Make is a tool primarily used for automation of code compilation but it can be used for all sorts of automation as we'll see. So let's see it in action.

Building your makefile

make executes a makefile that contains a list of make targets. So let’s start with something simple. Here is a very basic structure of a makefile target:

name: 
    task

It consists of a target name and a task associated with it.

Let’s write a real make target we can use to run our unit tests:

unit-tests:
    vendor/bin/phpspec

If we would run this target with make unit-tests it would run our phpspec tests (same as if we typed vendor/bin/phpspec).

Suppose we have another target for our functional tests:

functional-tests:
    vendor/bin/behat

Now we can use make functional-tests to run our functional tests.

But what if we want to run all our tests? Well, make targets can have a list of dependencies, targets that need to be run before it:

name: dependency dependency

In our case, we can create a new target and list our test suite targets as its dependencies:

tests: unit-test functional-test

Now, when we type: make tests that will run our unit-test and functional-test targets.

And what about those multiple steps tasks? Well, make targets can have more than one task associated with it:

name:
    task
    task
    task

So let’s create a make target to set up our (simple) project:

build-project:
    docker-compose up -d
    docker-compose exec app-comp composer install
    docker-compose exec app-php php artisan migrate

Advantages of using good makefiles

A good makefile provides us with some extra advantages:

we have a list of aliases for all the tools/tasks we use on a project that is shared between developers (and can be shared across projects also)
we have a single CLI entry point for our app so no more scouring documentation to find the name/syntax of some tool
developers don’t need to know the syntax of all tools used on a project— e.g. if you are a backend developer and setting up your project locally requires the usage of some frontend tools, having that wrapped into a make target would be helpful
multi-step tasks are automated and require a single command to be run

For clarity, using makefile targets doesn’t mean we never touch the underlying tools. We can still use those, especially when we need to run them with some specific configuration. Using make simply gives you a fast way to run them in standard configuration.

Wrap up

Now, in the beginning, I promised a nice Command Line Interface that will give us a list of all make targets that we can run. Well, this doesn’t come with makefiles out of the box, but with the help of a designated help target, makefile comments, and some bash magic we will have it working in no time.

First, we add comments to our make targets:

tests: test-unit test-functional ## Run all tests on the project

As you guessed, comments are prefixed with ##.

Next, we create a special help target that will render a list of our targets with comments as descriptions. There are different ways to achieve this, and you can look for some of them here. I will use the one from user muhmi:

help: ## This help dialog.
    @IFS=$$'\n' ; \
    help_lines=(`fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##/:/'`); \
    printf "%-30s %s\n" "target" "help" ; \
    printf "%-30s %s\n" "------" "----" ; \
    for help_line in $${help_lines[@]}; do \
        IFS=$$':' ; \
        help_split=($$help_line) ; \
        help_command=`echo $${help_split[0]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        help_info=`echo $${help_split[2]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        printf '\033[36m'; \
        printf "%-30s %s" $$help_command ; \
        printf '\033[0m'; \
        printf "%s\n" $$help_info; \
    donev

As you see, there is some bash wizardry here that I won’t pretend to understand fully, but it gets the job done and if you’re a bash wizard yourself you can make your own cool help target.

We also need to declare this help target as a default target so it would be run by default (if we run make without specifying any target’s name).

.DEFAULT_GOAL := help

Finally, we need to add a special line to bypass some traditional behavior of make. Since make is used for compiling files, if there was a file named “tests” in the folder our command wouldn’t run. This is avoided by using declaring the command as phony. “A phony target is one that is not really the name of a file.” By adding the .PHONY line and declaring all of our commands, this conflict will never occur.

Now our makefile is ready so let’s see how it looks:

.DEFAULT_GOAL := help
.PHONY: help build-project unit-tests functional-tests tests start stop
help: ## This help dialog.
    @echo "Hello to the AwesomeProject\n"
    @IFS=$$'\n' ; \
    help_lines=(`fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##/:/'`); \
    printf "%-30s %s\n" "target" "help" ; \
    printf "%-30s %s\n" "------" "----" ; \
    for help_line in $${help_lines[@]}; do \
        IFS=$$':' ; \
        help_split=($$help_line) ; \
        help_command=`echo $${help_split[0]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        help_info=`echo $${help_split[2]} | sed -e 's/^ *//' -e 's/ *$$//'` ; \
        printf '\033[36m'; \
        printf "%-30s %s" $$help_command ; \
        printf '\033[0m'; \
        printf "%s\n" $$help_info; \
    done
build-project: ## Build our project
    docker-compose up -d
    docker-compose exec app-comp composer install
    docker-compose exec app-php php artisan migrate
start: ## Start project containers
    docker-compose up -d
stop: ## Stop project containers
    docker-compose down
unit-tests: ## Run unit tests
    vendor/bin/phpspec
functional-tests: ## Run functional tests
    vendor/bin/behat
tests: unit-test functional-test ## Run all tests.DEFAULT_GOAL := help

If we would run make in a folder containing the makefile we just built, we would get our nice list of targets with their descriptions:

Hello to the AwesomeProject
target                         help
------                         ----
help                           This help dialog.
build-project                  Build our project
start                          Start project containers
stop                           Start project containers
unit-tests                     Run unit tests
functional-tests               Run functional tests
tests                          Run all tests

So what are you waiting for? Go supercharge your project with make. It is an easy way to create a consistent command interface so that ramp up time on projects is kept to a minimum.

What we talk about when we talk about testing

konrad_126 — Mon, 27 Apr 2020 19:52:03 +0000

Imagine if airplane companies never tested their airplanes. They just ship them when they’re done. Would you feel safe boarding a plane like that?

Fortunately, airplanes are not built like that. Flying is statistically the safest way of transportation and there are strict regulations and tests airplanes must abide by.

Most people would agree to release a product without testing it is a crazy idea, whether you’re building airplanes, cars, or bicycles. We expect a product must satisfy a strict set of tests before being released to the public. If a company releases a product that turns out to be faulty, it is subject to public ridicule. The first question that comes to mind is “Did they even test their product?”

What about software? For some reason, many people still think it is ok to build software without testing it

Someone will test it

When people talk about testing, what they usually talk about is automated tests, but let’s make one thing clear — every software program gets tested. Eventually, someone will run it and verify it’s output. If you are shipping software without testing it, you are letting your customers test it. As we saw with our airplane analogy, this is not a good idea, and hopefully, this is not what you are doing.

So you do test your software, but you just do it manually. You write the code for your feature and run your application. Then you go through a set of use cases or workflows to check if the application behaves as expected for each of them. Once it does, you ship it.

So what’s wrong with that? Why would you switch to automated tests?

Why automated testing?

Tell me if this sounds familiar. You write your feature and start checking if your application works as expected. But it doesn’t. There’s a bug so you fix it. You check again but find another bug so you fix it. After going through several (many) of these iterations everything finally works and you ship your code. As you are getting ready for lunch your error reporting tool goes haywire. Turns out, when doing your last round of workflow check-ups you skipped one and missed a bug.

Manual testing is repetitive work and the problem with us humans is that we tend to forget things. Luckily, there are these things called computers that are very good at repetitive tasks. Why not use them to test our application? Once we give the computer our list of workflows to check, it will never forget to go through any of them.

Another problem with manual testing is it takes time. The more workflows you have to check the more time it will take. In a fairly complex software system, the number of possible paths that an application flow can take grows fast, and testing them manually quickly becomes impractical or simply impossible. So you resort to testing only some of them. Computers are much faster at processing information than humans so they solve this issue too, allowing us to test our applications at dramatically greater speeds. The initial effort of writing extra code (an automated test) quickly pays off since the tests run before every push to a branch, deploy, or commit.

How?

If you do decide to start writing automated tests, you may get overwhelmed with all the different terminology and jargon used. In its essence automated software testing is simple — you write a program to test your program. So if your program is a calculator that does addition, you write another program that sends some input to your calculator. For example, we send numbers 2 and 3 and check if the output of the calculator matches our expectations (it should be 5). In reality, we are writing more complicated things than simple calculators but most of the time you will be testing if your program gives the expected output for a given input.

Other things you will often want to test are object/service collaboration and side effects. For example, you may want to know if your user registration service “told” the mailer service to send a welcoming email to the new user or if the user object was persisted in your data storage.

Keeping it efficient

As we pointed earlier, computers are much faster than humans, allowing us to test our applications with much greater speed, but as the number of tests rises, running (and maintaining) your tests can start to take time. This is why you want your test suite to follow a so-called Testing pyramid pattern.

The base of your pyramid is the low-level unit tests. Unit tests target single units in isolation (unit usually meaning a single class or path). As you go higher up the pyramid, your tests start to involve bigger “chunks” of your applications (components and modules) and with the highest level of tests booting up your whole application and testing it as a black box. The higher up the pyramid you go, the tests become slower and more brittle (making them harder to maintain) which is why you want a pyramid shape instead of a cone.

If you think about our airplane manufacturer analogy again, I don’t suppose they build the whole airplane and fly it just to check if the doors on the toilet work? This can be tested in isolation, as well as many other (more crucial) parts of the airplane like motors. Once they test each unit they can start assembling them into components and modules, testing them along the way, and finally ending up with test flights that test the airplane as a whole.

Tests first?

What if I told you that you should not only write tests but you should write the test first before you write the actual working code? This is what the practice of Test-Driven Development (TDD) is about. You write your test code and then the working code that will satisfy that test. This often seems like a counter-intuitive idea to many people, especially if they are not writing tests at all. After all, how can you test something that you haven’t built yet?

It is useful to think of your test code in terms of specification. Having a specification that defines how a certain part of the software should behave is a much more intuitive idea, and in a way, your test is just that — a specification of how certain parts of your software should behave. If you think of it this way then writing tests (specifications) before writing the actual software code makes much more sense. After all, how can you write software if you didn’t specify how it should behave?

Do I need it?

Learning how to write good automated tests will take some time and effort, so do you really need it? I mean, it’s not like you’re building airplanes, right?

Let me rephrase the question from the introduction. Would you board an airplane whose software (that it heavily relies on to fly) is not tested?

Maybe your web application failing won’t have the same consequences as airplane software failing, but as a professional, you vouch for the correctness and reliability of your software. While I am not claiming automated tests will make your software error-free, they are the best tool we have to raise the confidence in the correctness of our software. Unless constant failing is a requirement of your software, you need to write tests, because testing is too important to be left to humans.

Never talk to strangers

konrad_126 — Sun, 26 Apr 2020 18:30:49 +0000

Talking to strangers in 1930s Moscow was not a smart thing to do. Mikhail Alexandrovich Berlioz (chairman of the management committee of one of the biggest Moscow literary clubs — MASSOLIT) and his young companion, poet Ivan Nikolayich Poniryov (who wrote under the pseudonym of Bezdomny) knew that all too well. Yet, this is exactly what they did at the sunset hour of one warm spring day at the Patriarch’s Ponds when a suspicious-looking stranger approached them.

The stranger turned out to be the Devil himself and if you wish to know more about what awaits Berlioz, Bezdomni, and the citizens of Moscow when the Devil and his entourage come for a visit, go read Mikhail Bulgakov's masterpiece Master and Margarita. Since this is a blog on software development, you may be wondering: what does this have to do with software development?

Well, talking to strangers can be dangerous when it comes to software development, too. And it is expressed by an Object-Oriented design guideline called The Law of Demeter. It says that a "module should not know about the innards of the objects it manipulates". [1]

The formal definition of the Law of Demeter states that when writing a method in a class, our methods should only call:

other methods of the class
methods on an object that was created in our method
methods on an object that was passed as an argument to our method
methods on an object that is a class property

class TheClass {
    private OtherClass $instanceVariableObject;
    private function otherMethodOfTheClass() {}
    private function demeter(PassedInClass $argument)
    {
        // other methods of the class
        // this is okay
        $this->otherMethod();

        // methods of an object we created
        // this is okay
        $object = new ClassC();
        $object->someMethod();

        // methods on an object passed in
        // this is okay
        $argument->someMethod();
        // methods of a class property
        // this is okay
        $this->instanceVariableObject->someMethod();
    }
}

But, our method should not invoke methods on objects that are returned by any of these allowed method calls. In other words, objects should only talk to their friends, never strangers. And in this context, friends of friends are considered strangers.

Here is a typical Law of Demeter violation example:

$blog->comments()->add($comment);

Chained method calls can be a good indicator of this, but even if we “unchain” the methods, a violation persists.

$commentCollection = $blog->comments();
$commentCollection->add($comment);

Our calling code knows about the Blog object. This is its friend. The Blog object knows about the CommentCollection object. That is its friend. But to our calling code, CommentCollection is a stranger and we should not be talking to strangers. In more technical terms, our calling code is aware of the internal structure of the Blog class, so if this internal structure changes we would have to update our code. This is more coupling than we need.

What we can do instead is have a method on the Blog itself:

$blob->addComment($comment)

Our calling code only needs to know how to call this addComment method on a Blog object. It doesn’t need to know how a Blog object will add that comment internally, and if we change the way that a Blog keeps its comments internally, our calling code won’t need to be updated. Thus, we have reduced the coupling of our code. And that is good since higher coupling increases the chances that any (small) change we make will break something else. So we want less coupling.

The downside of obeying the Law of Demeter is that you may end up with a lot of delegate methods that act like wrappers:

public function addComment(Comment $comment)
{
    $this->commentCollection->add($comment);
}

Some would argue it could also be an indicator of problems in your design, especially if you have these methods nested across objects.

In any case, although talking to strangers in software development may not be as dangerous as in Soviet Russia (and you won’t end up like poor old Berlioz), The Law of Demeter is still a useful design principle to know and to apply when possible.

Martin Fowler

@martinfowler

@jeremydmiller I'd prefer it to be called the Occasionally Useful Suggestion of Demeter.

15:37 PM - 29 Apr 2009

15 16

[1] – Robert C. Martin – Clean Code

Understanding Git - Data Model

konrad_126 — Thu, 23 Apr 2020 16:27:11 +0000

As software developers, we use a lot of tools. Many of them come with an intuitive User Interface which abstracts the internal complexity of the domain they are operating on. This allows us to go fast and learn those tools by using them.

For better or worse, git is not one of those tools. The abstraction git's User Interface gives you is very leaky so to become better with git you must spend (some) time learning how it works internally.

In this Understanding Git series, we will cover git’s internals (we will not go into git’s source code don’t worry) and the first thing on that list is git’s heart and soul — the data model.

It's all about .git

We'll start by initializing a git repository:

git init

Git tells us it has created a .git directory in our project’s directory so let’s take a quick peek into it:

$ tree .git/

.git/
├── HEAD
├── config
├── description
├── hooks
│   ├── applypatch-msg.sample
│   ├── commit-msg.sample
│   ├── post-update.sample
│   ├── pre-applypatch.sample
│   ├── pre-commit.sample
│   ├── pre-push.sample
│   ├── pre-rebase.sample
│   ├── pre-receive.sample
│   ├── prepare-commit-msg.sample
│   └── update.sample
├── info
│   └── exclude
├── objects
│   ├── info
│   └── pack
└── refs
    ├── heads
    └── tags8 directories, 14 files

Some of these files and directories may sound familiar (particularly HEAD) but for now, focus on the .git/objects directory. Right now it's empty, but we will change that in a moment.

Let’s add an index.php file

touch index.php

fill it with some content

<?php
echo "Hello World";

and a README.md file

touch README.md

with some content as well:

# Description
This is my hello world project

Now let’s stage and commit them:

git add .
git commit -m "Initial Commit"

OK, nothing special here, adding and committing — we’ve all “been there, done that”.

If we look back at the .git directory we can see that the .git/objects directory now contains some files and subdirectories (bear in mind they will have different names on your computer!):

├── objects
│   ├── 5d
│   │   └── 92c127156d3d86b70ae41c73973434bf4bf341
│   ├── a6
│   │   └── dbf05551541dc86b7a49212b62cfe1e9bb14f2
│   ├── cf
│   │   └── 59e02c3d2a2413e2da9e535d3c116af1077906
│   ├── f8
│   │   └── 9e64bdfcc08a8b371ee76a74775cfe096655ce
│   ├── info
│   └── pack

Every object in git has a so-called checksum header (the unique identifier of an object) and the first two characters of that checksum are used as a directory name while the rest is used as a file (object) name. Let's look at what these objects are.

Blobs, trees, and ...

The first kind of object that git creates when we commit some file(s) are blob objects. Git uses them to represent the content of files. In our case there is two of them, one for each file we committed:

They contain the full content of our files, so you can think of them as snapshots of our files (at the time of the commit). To generate the checksum header git takes the content of an object, feeds it to a hashing function and the output is the checksum header. This is why it also serves as a unique identifier of an object.

The next kind of object git creates are tree objects. They are used to represent the project's folder structure and in our simple example, git needs only one tree object. It contains a list of all files in our project with pointers to their blob objects:

Lastly, git creates a commit object. It contains some metadata data (author, time..) and a pointer to its tree object:

Content of our .git/objects directory it should make more sense now:

├── objects
│   ├── 5d
│   │   └── 92c127156d3d86b70ae41c73973434bf4bf341
│   ├── a6
│   │   └── dbf05551541dc86b7a49212b62cfe1e9bb14f2
│   ├── cf
│   │   └── 59e02c3d2a2413e2da9e535d3c116af1077906
│   ├── f8
│   │   └── 9e64bdfcc08a8b371ee76a74775cfe096655ce
│   ├── info
│   └── pack

Using git log we can see our commit history:

commit a6dbf05551541dc86b7a49212b62cfe1e9bb14f2 
Author: zspajich <zspajich@gmail.com>
Date:   Tue Jan 23 13:31:43 2018 +0100Initial Commit

By knowing the naming convention we mentioned earlier we can locate this commit object in .git/object :

├── objects
│   ├── a6
│   │   └── dbf05551541dc86b7a49212b62cfe1e9bb14f2

To display its content we can’t simply use the cat command since these are not plain text files but git has a cat-file command we can use instead:

git cat-file commit a6dbf05551541dc86b7a49212b62cfe1e9bb14f2

Now we see the content of the commit object:

tree f89e64bdfcc08a8b371ee76a74775cfe096655ce
author zspajich <zspajich@gmail.com> 1516710703 +0100
committer zspajich <zspajich@gmail.com> 1516710703 +0100Initial Commit

First line is the pointer to a tree object and to examine it’s content we can use git ls-tree command:

git ls-tree f89e64bdfcc08a8b371ee76a74775cfe096655ce

As expected it does contain a list of our files with pointers to blob objects:

100644 blob cf59e02c3d2a2413e2da9e535d3c116af1077906 README.md
100644 blob 5d92c127156d3d86b70ae41c73973434bf4bf341 index.php

Let's look into the blob object representing index.php using the cat-file command:

git cat-file blob 5d92c127156d3d86b70ae41c73973434bf4bf341

Sure enough, it has the same content as our index.php file:

<?
echo "Hello World!"

There you go. Now you know what happens when committing files.

Let's see what happens if we now edit (add some code magic) and commit our index.php file:

Git creates another blob object (a new snapshot) to represent the new content of index.php. As for README.md, since it didn't change, git can reuse the existing blog for it (we'll see it in a moment).

When creating the tree **object, git updates the pointer for index.php to its new blob while the README.md pointer stays the same:

As before, the commit object has a pointer to the tree object but also a pointer to its parent commit object because every commit except the first one has at least one parent:

Now that we know how git handles file adding and editing, the only thing remaining is file deletion. What if we delete our index.php file?

It’s rather simple — git deletes the file entry (filename with a pointer to its blob object) from the tree object. In other words, our commit’s tree object no longer has a pointer to a blob object representing index.php (but that blob object is still there in .git/objects)

Nested folders

In our real-life projects, the folder structure is much more elaborate than in this simple example. As said, tree objects represent the folder structure of your project, and the same way folders can be nested tree objects can also be nested (point to other tree objects). For example:

Here, our project base folder has one README.md file and one sub-directory app which has two files ( app.php and app_dev.php).

So there you have it - git's data model. Blob objects represent the content of files, tree objects represent the folder structure of the project, while commit objects contain metadata and have pointers to their parents.

In the next post, we'll take a look at branching - what branches are and why having a bunch of them is very cheap in git.

git reset --explain

konrad_126 — Tue, 08 Jan 2019 20:32:09 +0000

Reset is probably one of the least understood git commands with the addition of having a bad reputation for being dangerous. There is a valid reason for both of these claims: yes, the reset command is a bit harder to understand and in some cases, it can be dangerous. But, it is not all that hard. So in this post, I will give my best to present you with a clear and distilled tutorial to the reset command. To make it short and not too overwhelming I have abstracted the non-essential details and simplified some things, but if you want to know more on git's internal workings you can also check my Understanding Git series for more details of some stuff presented here.

Git Trees

Before we dive into the reset command we need to take a look at what is called the trees of git: Working Directory, Staging Area and the Repository.

You can think of them as areas where changes can reside from git's point of view:

Working Directory - your project files on your filesystem
Staging Area - a preview of the next commit
Repository - datastore where git keeps all (past) commits.

The reset command operates on these threes/areas, but first, let's take a look how do add and commit command that we use daily, affect these areas.

Say we have a web app and we do some refactoring on our index.php file. Changes that we make are reflected in the Working Directory:

We can confirm this by running git status:

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
modified:   index.php

Now we move those changes to the Staging Area by using the add command:

And running the status command now will tell us:

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)
modified:   index.php

Because the status command sees that we have the same version of index.php file both in Working Directory and in the Staging Area, but not in Repository.

To add it there, we use the commit command:

And now the Working Directory, Staging Area and Repository all contain the same version of index.php, and running git status will tell us that there is:

nothing to commit, working tree clean

So, the way the status command works is that it compares file versions in Working Directory, Staging Area and Repository and if there are different than there are files to be staged/committed.

Let's say we now refactor the index.php file some more and do the whole add/commit cycle again.
Now our Working Directory, Staging Area and the Repository all contain the new second version of our index.php file.

But what about the first version? If you remember, we did say that the Repository keeps all previous commits, so the first version of index.php file is still there:

To keep track which version of our index.php file is the current one, Repository has a special pointer called HEAD that points to the current version (and the status command only looks at the current version that HEAD is pointing to when comparing it to the Staging Area's version).

And now that we have that covered we can finally go to our reset command and see how it works by manipulating the content of these areas.

reset --soft

This first mode of reset command will only do one thing:

move the HEAD pointer.

In our case, we will move it to the previous commit (the first version of index.php) by doing: git reset --soft HEAD~1

The trees of git now look like this:

And if we would to run git status we would see a familiar message:

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)
modified:   index.php

So, running git reset --soft HEAD~1 has basically undone our last commit, but the changes contained in that commit are not lost - they are in our Staging Area and Working Directory.

reset - mixed

The second mode of the reset command will do two things:

move the HEAD pointer
update the Staging Area (with the content that the HEAD is pointing to)

So, the first step is the same as with the --soft mode. The second step takes the content that HEAD points to (in this case, it is version one of the index.php file) and puts it into the Staging Area.

So, after running git reset --mixed HEAD~1 our areas look like this:

And if we would to run git status now we would again see a familiar message:

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
modified:   index.php

So, running git reset - mixed HEAD~1 has undone our last commit, but this time the changes from that commit are (only) in our Working Directory.

reset --hard

And now for the notorious hard mode. Running reset -- hard will do three things:

move the HEAD pointer
update the Staging Area (with the content that the HEAD is pointing to)
update the Working Directory to match the Staging Area

So, the first two steps are the same as with --mixed.The third makes the Working Directory look like the Staging Area (that was already filled with what HEAD is pointing to).

So, after running git reset --hard HEAD~1 our areas look like this:

And running git status will give us:

nothing to commit, working tree clean

So, running git reset - hard HEAD~1 has undone our last commit and the changes contained in that commit are now neither in our Working Directory or the Staging Area. But they are not completely lost. Git doesn't delete commits from Repository (actually, it does sometimes, but rarely), so this means our commit with the second version is still in the Repository, only it is a bit harder to find (you can track it by looking at something called reflog).

So, what is then with this reputation of reset being dangerous? Well, there is one case where some changes could be permanently lost. Consider a scenario where after the second commit you make some more changes to your index.php file, but you don't stage and commit them:

And now you run git reset --hard HEAD~1:

Since the reset command will overwrite the content of your Working Directory to match the Staging Area (that is made to match HEAD) and you never staged and committed your changes (there is no commit with those changes in the repository), all those changes will now be lost in time... Like tears in the rain.

The danger of hard reset lies in the fact that it is not Working Directory safe - meaning it won't give you any warning if you have file changes in your Working Directory that will be overwritten (and lost) if you run it. So be (extra) careful with a hard reset.

And there you have it: the reset command. I hope I did a good job explaining it and that you'll agree it is not that hard after all. And, yes it can be dangerous but only if used with --hard option.

As said in the beginning if you would like to know more on the inner workings of git, you can check my Understanding Git series, and if you want a more in-depth explanation on the reset command you can check the Reset Demystified chapter from git pro book.

Appendix:

In the examples, we used we used HEAD~1 as an argument for the reset command. As you probably already know every commit in git has a unique identifier called a checksum, and we can use it as an argument for the reset command too

To make examples simpler we only had one file that we edited and committed, in reality, we often commit multiple files, so a specific commit holds different versions of multiple files.

The special HEAD pointer usually doesn't point directly to a commit (as shown in examples for simplicity) but to a branch pointer that then points to a specific commit