What is magic in code and why avoid it

I have been building software for more than 8 years now and one lesson I have learnt the hard way was to avoid "using magic code" as much as possible.

So... what a strange statement, right ? "Avoid magic" ? What does that even mean ?

Please allow me to explain what these words mean to me.

What is magic in code

What I refer to as "magic code" is the concept of something happening inside your software without a direct, meaningful statement written by you, the developer.

Here is an example:

databaseManager = new DatabaseManager();
databaseManager->persist();

In this little snippet of code, I have written a direct function call persist(). Consequently I expect that the computer will perform a call to persist() when the snippet is run. The behavior and how the code can be read are closely related. I could read it aloud and what I would say and what happened would match.

Now what about this:

databaseManager = new DatabaseManager();
magician.call('persist', 'all');

Here is a first example of magic: the above object magician has the ability to look for all existing objects and call the chosen function on them. So the function persist() will be called, just like in previous snippet, however ; in the code above ; the relationship between what is written and what happens is not clear. The two snippets of code produce the same outcome, but they are written very differently.

A second example will be handy:

public class MyClass {
  int x = 5;

  public static void main(String[] args) {
    MyClass myObj = new MyClass();
    System.out.println(myObj.x);
  }
}

The class above is pretty straightforward to read. This is a class which holds an x integer property, initialized to 5. The main function will print it.

What about this trick:

public class MyClassWithXFive {
}

Believe it or not, but the empty class MyClassWithXFive behaves exactly like the above MyClass. It holds an x integer property, initialized to 5 and it has a main function that prints it.

This is because in some software I am working on, there is a rule (a magic rule ?) that when a classname ends with "WithXFive" then the software automatically generates a x property initialized to 5 in it! This property can be used just the same as the previous snippet.

This is a typical example of magic behavior in code: some properties, some definitions, some rules that happen not because of the code you have written but rather because of the names you have chosen for your classes or properties.

Some more examples of the same kind:

• declaring a property id and this property is automatically used as main identifier for given class
• declaring a class with suffix Test and this class is automatically considered and run as a test

Different types of magic

Here is a not-exhaustive list of what I would consider as "magic" in the code:

1) As mentioned before, behaviors that happen because of the name of your classes, properties, files

This kind of magic would happen thanks to a parser that would extract your classes / properties / files names and then look for specific patterns. Upon finding such pattern, it would trigger some behaviors.

This means that the name of your classes is determining for your application behavior. When modifying the name of your classes you might then introduce bugs.

2) Behaviors that happen because your code is located into a specific directory

This kind of magic would happen thanks to a finder that would look at a given directory and trigger some behaviors on all findings of this directory.

This means that the location of your classes is determining for your application behavior.
Moving your source code files might then introduce bugs.

3) Artifacts automatically generated

This kind of magic parses your code and uses it to build other chunks of code that are computed and written on disk. The code that is, eventually, processed at runtime is the generated code, not the one you have written.

This one is the less dangerous of all and it is used quite often for cache purposes.

---

Ironically, there is a concept called magic strings but I would not consider it to be "magic" 😄 it's another kind of dangerous magic though.

Why software projects use magic

I believe that most of the time, magic is being built into a software for the purpose of simplicity. It allows the developer to avoid writing some steps, some artifacts, some boilerplate code. It reduces duplication and "no brainer" code to write. When I tell you that "the only thing you need to do is to add your file in folder Z and you're done", it sounds appealing, right ?

"Magic code" might appear, at a given time, as an elegant solution to a real problem. There are well-known best practices in software that urge us not to waste time on meaningless tasks. The "Automate everything" moto is one of them.

Magic code looks like a good solution to such problems at a given time. The drawbacks, however, come later.

The issues of magic code

The reason I am strongly suggesting to avoid magic as much as possible is how hard it is to deal with it when things go wrong.

In order to illustrate my statement, please allow me to share a story, starring Bob, a random developer.

Story starts, Bob is hired to work on a codebase where there is one magic behavior implemented. This codebase is a web application, and Bob is told by the team leader that, although the application is expected to be used in a browser by most users, it does provide a REST API. Good news: the REST API is built in a very smart way: it's generated by a system that automatically expose all entities whose name ends with "Exposed".

So in the codebase, Bob looks at some entities:

• There is an entity UserExposed, and because of its name, it is consequently in the scope of the REST API. Indeed Bob can see there is a /users REST API endpoint available on the application.
• ProductExposed is in the scope too, there is a /products API endpoint Bob can use too.
• There is a Supplier entity and it is not in the scope, because its name does not match the rule.
• There is also a Brand entity, and it is not in the scope either.

Now that Bob is more familiar with the codebase and how the API is built, here comes the Product Owner. He asks Bob to create two new endpoints in the REST API: /suppliers and /brands. Cool, it seems quite easy, right ? Bob only needs to rename the entities to SupplierExposed and BrandExposed and "everything will be done for you by the API system". Sounds magic!

So Bob starts working, opens his IDE and modifies the classnames. Bob starts with Brand and he renames the class to BrandExposed. Oh! The API endpoint /brands is immediately available, even the documentation is up-to-date! It can immediately be used, and Bob did not need to write a single new line of code. This is truly awesome!

Then Bob modifies Supplier and rename it to SupplierExposed. Bob saves the file, commits the changes. But... nothing happens. Weird. The API endpoint should be implemented. Did he miss something ? He double-checks and double-saves the file, waits a little... But still no /suppliers endpoint. Nothing. Nothing comes out of the void.

And this... is where the bad things start. Because Bob needs now to understand why it does not work (and fix it). 😱

Because this system is "magic" it is very hard for Bob to track where the functions in charge of the REST API generation are. It's very hard to find why something is not triggered. Bob does not even know when it's supposed to be triggered.

In traditional imperative programming statements, you can follow easily how functions and changes occur. Debugger can even print it out for you! But now that magic is in the place, you (and Bob) cannot rely on it anymore. Magic code makes your software hard to navigate.

Let's say that after one hour and some sweat, Bob has found the area in the codebase responsible for the REST API endpoints generation. It gets worse: the same code is responsible for generating /products, /users, /suppliers endpoints! So the code is full of things like generateName(), loopThroughProperties, loopThroughEligibleObjects(). The code is very abstract, it's hard to track what's going on and relate it to the bug.

And if Bob finally finds what he believes to be the root cause of the issue, he has yet to make sure that his changes do fix the /suppliers endpoint without modifying/breaking all the other endpoints, generated by the same code! 😱

I think the story of Bob illustrates well the pitfalls of magic in code: generic code, abstract code, hard to relate with an actual bug scenario, and hard to navigate, debug and maintain.

This is, by the way, one of the drawbacks that is sometimes mentioned to discourage usage of Frameworks. It's not unusual for frameworks to rely on magic. As mentioned before, this can become a very handy asset in the hands of developers, but when there is a need to debug/expand/update the magic behavior, things can get nasty.

So avoid magic code

Magic code is a double-edged sword. It has tremendous potential and can allow building very powerful systems. It can also boost performances in some situations, like caching.

However these benefits come with a (very) high maintenance cost, which is why I advise you to avoid it in most situations.

Following the blog post I wrote 3 years ago "Basic code is easy to maintain" I still believe that most of us should choose simplicity over efficiency.

Unless you're Google or Facebook. Then you can choose efficiency and complexity 😁 and you'll be fine!