DEV Community: Marco Mannucci

DevTools tips: Copy request as code

Marco Mannucci — Wed, 30 Sep 2020 22:09:21 +0000

Sometimes I need to replicate a network request made by the browser: maybe I want to test it directly without fiddling with the website UI; maybe I need to embed it in some backend application to obtain the same resources or invoke the same API.

The network inspector in Chrome/Chromium makes a good job at showing all the nitty-gritties of what happens at the HTTP level, detailing the structure of the various request/response exchanges.

But extracting that information and translating in something usable for the purpose described earlier may result overly tedious and error prone.
Suppose I want to replay one of the requests as a cUrl command in my shell. I should:

examine the request to extract all the necessary data
read cUrl docs in order to find out how to build the desired request
actually build the request
test it 🤞

Usually, this also turns out to be a trial and error process.

For this purpose, DevTools come in our help!
If we right-click, from the network inspector, the request we want to replicate and expand the copy menu voice, we will se a lot of options.
Besides the classical operations for copying the path or the url of the request, we are offered a lot of other voices

In particular, there are a series of Copy as... commands which will put in our clipboard a ready-made snippet of code of the selected language to be used to execute the request!

For example, I can Copy as cUrl(bash) to get a nice command to use in my shell

curl 'http://localhost:8080/' \
  -H 'Connection: keep-alive' \
  -H 'Pragma: no-cache' \
  -H 'Cache-Control: no-cache' \
  -H 'DNT: 1' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'Sec-Fetch-Site: none' \
  -H 'Sec-Fetch-Mode: navigate' \
  -H 'Sec-Fetch-User: ?1' \
  -H 'Sec-Fetch-Dest: document' \
  -H 'Accept-Language: en,it-IT;q=0.9,it;q=0.8,en-US;q=0.7' \
  -H 'Cookie: *******' \
  --compressed

If I want to make the same request in vanilla Javascript, I could Copy as fetch and get

fetch("http://localhost:8080/", {
  "headers": {
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "accept-language": "en,it-IT;q=0.9,it;q=0.8,en-US;q=0.7",
    "cache-control": "no-cache",
    "pragma": "no-cache",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "none",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1"
  },
  "referrerPolicy": "strict-origin-when-cross-origin",
  "body": null,
  "method": "GET",
  "mode": "cors",
  "credentials": "include"
});

DevTools tips: Inline source editing

Marco Mannucci — Wed, 23 Sep 2020 16:53:20 +0000

One of the cool features of Chrome/Chromium DevTools, is the ability of modifying js scripts code directly in the source inspector!

Just open the Source panel and start hacking in the code of one the listed scripts.
Save your modifications with ctrl-s (cmd-s on Mac) and check the result in your app!

This may come in handy if you you want to try out simple modifications on the front-end without rebuilding the whole app or even refreshing the page!

Check it in action 👇

DevTools tips: Logpoints

Marco Mannucci — Thu, 17 Sep 2020 12:28:15 +0000

When it comes to debugging javascript, there is one single tool that consinstently pops out, no matter how sophisticated the browser debugger is: console.log()

🤨

Ok, it's not really elegant but sometimes is an effective way to monitor what's happening in our code without recurring to breakpoints and watch expressions and without the need to pause the execution of our script.

The drawback is that console.log() statements tend to clutter the code and are really there for testing purposes only: we should remove them once we are done but there's always the risk that they slip in some production code 😫.

Luckily, any Chromium based browser has a nice feature that takes the best of the two approaches: Logpoints.

These are quite similar to brakpoints in the way they are set, but they allow us to define a message in the same way we would do for a console.log() statement, even passing variable or executing functions visible in the current scope.
Plus, script execution won't stop on a logpoint but rather will print the defined message in console.

Logpoints only live in the browser, so no risk of forgetting them in the code, and we can enable, disable or remove them as if they were breakpoints.

Check it out!

Simple and effective!

A simple refactoring

Marco Mannucci — Thu, 10 Sep 2020 16:53:36 +0000

Refactoring is the act of taking a function, a method, a class or an entire application and modify it without altering its functionalities in order to make it better, more performant, more mantainable. To improve it in some way while mantining its original behavior.

I recently worked on the Metric/Imperial converter project on freecodecamp.org and had the opportunity to make a little refactoring.

I'd like to write down and share the passages that led from the initial implementation to the, hopefully better, last form.

I'm not claiming I've done a mirable work or I'm revealing some hidden truth. I'm simply trying to elaborate my thoughts on coding, share them and get feedbacks!

So, let's dive in 🏊‍♂️

You can find the specification on the project page, but the bit we are interested in is the following:

I can use fractions, decimals or both in my parameter (ie. 5, 1/2, 2.5/6), but if nothing is provided it will default to 1.

The input string should also include the unit we are converting from, so valid inputs are:

4gal
1/2km
5.4/3lbs
kg

The project blueprints make us implement a getNum(input) function whose goal is to parse the numeric part of the input. That's the function we will focus on.

Refactoring would not be possible without a suite of tests that guarantees we are not altering the behavior of our code and potentially adding bugs.

So here it is!

    test('Whole number input', function(done) {
      var input = '32L';
      assert.equal(convertHandler.getNum(input),32);
      done();
    });

    test('Decimal Input', function(done) {
      var input = '3.2L';
      assert.equal(convertHandler.getNum(input),3.2);
      done();
    });

    test('Fractional Input', function(done) {
      var input = '3\/2L';
      assert.equal(convertHandler.getNum(input),1.5);
      done();
    });

    test('Fractional Input w/ Decimal', function(done) {
      var input = '3.5\/2L';
      assert.equal(convertHandler.getNum(input),1.75);
      done();
    });

    test('Invalid Input (double fraction)', function(done) {
      var input = '3\/4\/2L';
      assert.throws(convertHandler.getNum.bind(convertHandler.getNum, input), 'invalid number');
      done();
    });

    test('No Numerical Input', function(done) {
      var input = 'L';
      assert.equal(convertHandler.getNum(input),1);
      done();
    });

Here we are adhering to the tests provided by the project. More cases and conditions could be tested but let's keep it simple.

My (really naive) first implementation is the following:

function ConvertHandler() {

  this.getNum = function(input) {
    var result;
    let match = /[a-zA-Z]/.exec(input); // Searching for the unit part
    if (match) {
      result = input.substring(0, match.index);
    }

    if (!result) {
      return 1;
    }

    if (result.indexOf('/') != -1) { // Parsing the fraction
      let operands = result.split('/');

      if (operands.length != 2) {
        console.log('throwing error');
        throw new Error('invalid number');
      }
      result = parseFloat(operands[0]) / parseFloat(operands[1]);
    }
    else if (result.indexOf('.') != -1) {
      result = parseInt(input);
    } else {
      result = parseFloat(input);
    }

    return result;
  };
}

It hurts the eyes (and I wrote it! 😖) and, though it adheres to the specifications and passes the tests, it's not really clear what's going on. Morever, modifying it when new requirements show up would not be trivial.

I wrote it following the requirements driven by tests. So, I added "features" in the order they were presented by the tests.
In retrospective, I think this approach quite mimics what generally happens on a codebase when new requirements pop out: the requirement is analyzed, you find a suitable spot to implement it, write some code trying to adapt to what is already there and make it work.

And that's perfectly fine, but once it works, we should take some time to reflect on what's going on and if there's any room for improvement (spoiler: there always is). It's not easy and it requires a lot of effort. In that sense, I think this kind of exercise is really useful for building a sort of "muscle memory" on refactoring.

In this specific case, it came to my mind that integers and decimal numbers are merely special case of a fraction.
So, from a functional point of view, we can generalize the method so that it handles fractions only.

We should only provide sensible defaults for numerator and denominator:

denominator = 1 when there is only an operand
numerator = denominator = 1 when no number is provided

So, let's try a second iteration

function ConvertHandler() {

  this.getNum = function(input) {
    var result;
    let match = /[a-zA-Z]/.exec(input); // Searching for the unit

    if (match) {
      result = input.substring(0, match.index);
    } else {
      throw new Error('invalid input');
    }

    let numerator;
    let denominator;
    let operands = result.split('/'); // Parsing the fraction

    if (operands.length > 2) {
      throw new Error('invalid number');
    }

    if (operands.length >= 1) {
      numerator = parseFloat(operands[0]);
    }

    if (operands.length == 2) {
      denominator = parseFloat(operands[1]);
    }

    result = (numerator||1) / (denominator||1)

    return result;
  };
}

Much better! 😃

The function now tries to parse a fraction by splitting on '/', check for how many operands are provided and applies the defaults by short-circuiting the variables (e.g. numerator||1)

Now we have cleared our mind on the function even the code turned out clearer: variables have more meaningful names, flow control has less branching and in general is easier to read the code.
The test suite guarantees that the behavior of the function is the same.

The function is still a little verbose, with lot of if statements and some room for inlining. We can take advantage of some features of the language to make the code more concise.

For example, we can leverage the fact that javascript won't complain if we access an array out of bounds, returning an undefined value instead:

function ConvertHandler() {

  this.getNum = function(input) {
    let match = /[a-zA-Z]/.exec(input); // Searching for the unit
    let numericString 

    if (match) {
      numericString = input.substring(0, match.index);
    } else {
      throw new Error('invalid input');
    }

    let operands = numericString.split('/'); // Parsing the fraction

    if (operands.length > 2) {
      throw new Error('invalid number');
    }

    return (parseFloat(operands[0]) || 1) / (parseFloat(operands[1]) || 1);
  };
}

Here I also inlined the parseFloat() invocation since I find there is no value in keeping the two variables for numerator and denominator.

One thing that really bothers me at this point, is the presence of operations like regex matching and string parsing. They are a little too low level and require a lot of brain power to figure out what their purpose is; also, the accompanying comments are a hint that something should be done to improve readability and comprehension of the code.

One technique that tackles this kind of problem is method extraction: we literally take pieces of our code and encapsulate them in external functions that we could invoke in place of the removed code.
So we can reason to a higher level, with the added benefit that we can name our functions in a more meaningful way, thus conveying the real intention of our code.

In this iteration I extracted the findUnitIndex(), extractOperands() and parseAndCoalesce() methods.

function ConvertHandler() {

  this.getNum = function(input) {
    const unitIndex = findUnitIndex(input);
    const operands = extractOperands(input, unitIndex);
    return parseAndCoalesce(operands[0]) / parseAndCoalesce(operands[1]);
  };

  /*
   * Extracted methods
   */
  function findUnitIndex(input) {
    const match = /[a-zA-Z]/.exec(input);
    if (!match) {
      throw new Error('invalid input');
    }

    return match.index;
  }

  function extractOperands(input, matchIndex) {
    const operands = input.substring(0, matchIndex).split('/');
    if (operands.length > 2) {
      throw new Error('invalid number');
    }

    return operands;
  }

  function parseAndCoalesce(operand) {
    return parseFloat(operand) || 1
  }
}

The resulting code in the main function is more concise and it's really easy to understand what's going on at high level.
Complexity is pushed down in the extracted methods, so we really didn't get rid of it. But we have isolated and "labelled" it, making it easier to intervene.

One last thing that I'd like to change is the line:

parseAndCoalesce(operands[0]) / parseAndCoalesce(operands[1]);

to make its purpose clearer.

What I came up to is a brand new concept: CoalescingFraction (I don't really know if that'a thing but I think we can get creative here).
The idea is that of a fraction that defaults its numerator and denominator to 1 in case they are not provided (it would have made more sense that numerator defaulted to 0, but we are following the project specification).
The technique used here is class extraction: we encapsulated an entire concept in a new class, pushing it away from our main code and also making it available to other parts of our application.

function ConvertHandler() {

  this.getNum = function(input) {
    const unitIndex = findUnitIndex(input);
    const operands = extractOperands(input, unitIndex);
    return new CoalescingFraction(operands[0], operands[1]).value();
  };

  /*
   * Extracted methods
   */

   // as previous step; redacted for readability

}

/*
 * Extracted class
 */
function CoalescingFraction(numerator, denominator) {

  this.value = function() {
    return parseAndCoalesce(numerator) / parseAndCoalesce(denominator);
  }

  function parseAndCoalesce(value) {
    return parseFloat(value) || 1
  }
}

More inlining is possible, but I think that is good enough.

Conclusions

Refactoring is something I really should do more often and this kind of little exercises are a good way to make practice and explore the possibilities in a safe environment.

Tests are important in order to guarantee the correct outcome. However, I think one should find the right granularity: having too many of them to mantain could indeed make refactoring harder.

When resolving a problem, we often stop once a solution is found. Taking some time to think harder on it may lead to better results that are both more effective and pleasant to our mind.

Hope you enjoyed the post! 😊

Use bind to assert a function throws error

Marco Mannucci — Sun, 06 Sep 2020 22:20:23 +0000

If you are using Chai (but that should be valid for other testing frameworks) for your unit tests and you need to assert that a function throws an error, you can use the assert.throws function.

However, the doc is not really clear on how to use it so, if you are testing a function foo to throw an error with input bar you could be tempted to write the test like this (as I initially did):

assert.throws(foo('bar'), "Expected error message");

Turns out this way the test does not work.

Reading more carefully the documentation, we can see that the assert.throws function expects, as its first argument, a function

.throws(fn, [errorLike/string/regexp], [string/regexp], [message])

but in my first attempt I was passing the application of a function.

One possible way to make the test function as it is supposed, is to use the bind method:

assert.throws(foo.bind(foo, 'bar'), "Expected error message");

It will create a new function with the this keyword set to the first argument, in this case the function itself but it could be whatever makes sense. Next arguments will be passed to the new function when it will be actually called, so we can provide the input under test.

Chai can now use the result of the bind function, a new function, to execute the test in the way it is supposed to, passing the input that should verify our assert.

How do you cope with database migration?

Marco Mannucci — Thu, 03 Sep 2020 23:19:39 +0000

I find databases (mostly the relational one) to be one of the components more adverse to change in a software system. That's even worse if there some kind of ORM between the tables and the data model in our application.

There's always some resistence in altering a database, especially in a consolidated product that is already live in production.

With database migration I mean every operation needed to handle the porting of a database from its current version to some new version, maybe after introducing some new feature in the app that impact the data schema.

So I'm talking about versioning, scripts, data import and update, dedicated automation etc.

I'm a long time user of Liquibase for this kind of stuff, but I find it a little cumbersome in its usage and painfully slow when there are lot of migrations to analyze (I'm using it in it's Groovy flavour, maybe it depends on that).

Recently I explored Active Record migration and found it quite refreshing, although I think it's a bit tied to Rails (where Liquibase it's more obiquitous).

I'm curious to hear what are your tools and strategies to cope with database migration.

On regex capturing groups

Marco Mannucci — Thu, 27 Aug 2020 20:23:38 +0000

I always used capturing groups in regexes to extract parts of the string I was matching for later reference (for example, substitutions).

TIL that they can also be used during the actual matching 😮.

What the heck are capturing groups?

Given a piece of code with this method invocation:

doSomething(firstArg, secondArg)

if I want to make a substitution to use another method which needs the arguments swapped, I can do it with a regex like this:

doSomething\((.+),\s(.+)\)

The parts of the expression inside the round brackets (aside the escaped ones) are capturing groups. They can be used to treat the wrapped expression as a whole, for example applying a quantifier.
Moreover, capturing groups make it easy to extract part of the regex match.

In the previous example, after the match, I can refer to the extracted groups by their position.
So group 1 will be firstArg and group 2 willl be secondArg.

If my editor supports regex substitution or I'm writing some script to automate the operation, I can do something like this to replace the method:

doSomethingElse($2, $1)

which will result in

doSomethingElse(secondArg, firstArg)

(the right syntax really depends on the tool you are using).

Apparently there's more to that!

This kind of substitution is really cool, but capturing groups can be more powerful than that! 🚀

They can also be used during the matching itself to check if a part of the string matches another part already matched.
For example, if I want to check if a string starts and ends with the same sequence of numbers allowing any character string in the middle, I can do it with something like this:

(\d+)[A-Za-z]+\1

The first and only capturing group (\d+) will match any number sequence and the [A-Za-z]+ expression will match the character sequence.
The magic happens with the \1 expression, which is a way to reference the first capturing group of our expression, verifying that this last part of the string matches that group. The number corresponds to the number of the capturing group.

With this regex we can check for strings like:

123HelloWorld123

A more realistic scenario may be a csv file with records holding date of creation and update:

name,       created,    updated
record_1,   2020-04-10, 2020-04-21
record_2,   2020-04-11, 2020-04-11
record_3,   2020-04-12, 2020-04-22
record_4,   2020-04-13, 2020-04-13
record_5,   2020-04-14, 2020-04-20
record_6,   2020-04-15, 2020-04-16
record_7,   2020-04-16, 2020-04-16

We can use capturing groups if we want to find records whose created and updated dates are equals.
The regex will be something like this:

.+?,\t(\d{4}-\d{2}-\d{2}),\t\1

Not much readable but quite simple. The important bits are the capturing group in round brackets that matches a date and the \1 recalling it. The regex will match only if this two elements are actually equals.

Running the search in a text editor will highlight all the lines whose dates corresponds.

These are simple examples and more complex matchings are possible but it really blows my mind that a feature of regexes that I already considered really useful is indeed a lot more powerful!

Gradle daemon with multi module Spring project

Marco Mannucci — Tue, 25 Aug 2020 13:18:09 +0000

I'd like to share the way I'm handling a multi module project in Spring. In particular, how to start multiple web applications at once taking advantage of the Gradle daemon.

The problem

I've a multi module project which comprises different web apps that should run together.
I noticed that executing

gradle :application_A:bootRun

and, once application_A is built and running,

gradle :application_B:bootRun

two different Gradle daemons are spawn even if the two executions are eligible for using the same daemon.
You can check it with the command:

gradle --status

You should see something similar to

   PID STATUS   INFO
 23606 BUSY     3.5.1
 24703 BUSY     3.5.1

(I know, that's and old version of Gradle...)

This has a huge performance impact, since a gradle daemon is quite eager in term of resources and having different instances defies its very purpose.

Why does that happen?

The reason why the second Spring application is not using the existing daemon is that the bootRun task simply does not terminate: it remains in the running state until we explicitly kill the process. This behavior keeps the daemon in BUSY state so application_B is obliged to spawn a new daemon.

What can we do?

Assuming we don't want to disable the Gradle daemon (that of course will solve the problem...) or simply we can't (e.g. the Tooling API used by IDEs always starts a daemon), the first solution that comes to mind is to execute the two tasks as a single Gradle command, like that:

gradle :application_A:bootRun :application_B:bootRun

Unfortunately, this does not solve our problem... Gradle will gladly run application_A but once again will stop on that task until we don't terminate it. So, this time we have one single daemon, but also one single app started 🙄.

So?

Turns out that the latter is indeed the correct approach but we have to enable parallel execution, which will allow Gradle to run multiple task in parallel, without waiting for the previous task to end. In particular, it will try so execute :application_B:bootRun even if :application_A:bootRun is not terminated (our original problem).

This is as simple as adding the --parallel flag.

gradle :application_A:bootRun :application_B:bootRun --parallel

Now you should see both the apps up and running waiting for requests 😊.

In older versions of Gradle (like mine) this is not sufficient. We should also tell Gradle the number of workers it is allowed to use. Apparently, it otherwise sticks to 1 so that, even with --parallel flag, the execution is sequential.
We can easily solve this with the --max-workers flag:

gradle :application_A:bootRun :application_B:bootRun --parallel --max-workers=4