loading...
Cover image for Handling complex MVC applications -  How to scale and avoid Controller chaos

Handling complex MVC applications - How to scale and avoid Controller chaos

pavlosisaris profile image Paul Isaris Updated on ・7 min read
Post cover image provided via undabot.com

This article uses Laravel for the code snippets, but the paradigm can be easily adapted to every other MVC framework out there.

To make things more interesting, we will lay this article out by posting an imaginary conversation between 2 professionals:

Stan, a seasoned developer, who has made many architectural mistakes (but thankfully seems to be learning from them), and

Ollie, a novice developer, who just started delving into the world of serious programming and has some simple applications.

Here is how the conversation went:

Introduction

[Stan] Hey Ollie! Today I would like to talk to you about MVC. MVC is a popular Architectural Pattern for building robust applications.

[Ollie] Wait wait wait… MVC? What is that?

[Stan] MVC stands for Model - View - Controller. It essentially defines a strategy for robust code design. It breaks down the application to different parts, in order to achieve higher separation of concerns between the app's modules.
Having modularized code helps a lot when trying to add more functionality or maintaining it.
Let’s see the following picture:

MVC framework layout

Image taken from Wikipedia

[Stan] So, the user interacts with the View, and upon each interaction, relevant Controller methods are called, who then are responsible for updating/ fetching the appropriate data back to the user (again by using the View layer).

[Ollie] Gotcha! I have already used MVC in many projects. I know a ton about it!

The typical code example

[Stan] Not too fast, Ollie, there is a lot more to cover.
Now, let’s see a Laravel Controller method in a typical tutorial that we usually stumble upon:


<?php  
namespace App\Http\Controllers;  

class HomeController extends Controller {  

  public function simpleMethod() {  

    $books = Books:all();
    foreach($books as $book) {
        // some business logic here...
    }
    return view('home.home')->with(['books' => $books]);  
  }  
}

[Ollie] OK, this code seems pretty straightforward!

[Stan] What if we could make in better by breaking it down into individual components?

[Ollie] Why on earth should we wanna do that? I like that code; simple and it does the job. What is so special about it?

[Stan] There is something very special about it, my young Padawan! It’s called “Complex Applications”.
See, in a relatively small project, it is perfectly fine to have everything in a Controller method.

[Ollie] Wait wait… what do you mean when saying everything…?

Layers to the rescue

[Stan] I sense that you fail to understand how most applications work… In a typical MVC application, we have the following layers:

  • Validation Layer

  • Business Logic Layer

  • Database/ Repository Layer

  • Error Handling Layer

  • Success/ Error displaying layer

To be honest, an MVC application can be broken down further into even more layers, but these are the most basic ones. It’s perfectly fine if most tutorials omit them. It’s not their job to explain how to layer an application, but rather explain the topic that the tutorial is about.

[Ollie] OK… So why should I break down my application? If I want to add some functionality, I can do it in my Controller method.

Rotten code over time

[Stan] Of course you can go ahead and do this, but at some point it will result in code duplication , extremely large Controller methods and classes, and rotten code in general.

This is why I said that layering down works better in complex applications.

If you don’t plan to scale your project, then you may be fine leaving everything in your Controller method.

As an application grows, it becomes harder to maintain. As complexity increases, the value of reusable modules increases as well. We know that we have to do something about it before we risk technical debt, which in turn will result in harder maintenance and further development.

Let’s see the code from the previous example, after some time of adding more functionality;


<?php  
namespace App\Http\Controllers;  

class BooksController extends Controller {

    public function complexMethod(HttpRequest $request) {

        $authorId = $request->author_id;
        if(!Author::find($authorId))
            // wrong data
            return back()->with('error','Wrong data');
        try {

            // this DB query needs to be duplicated if we want
            // to use it in another part of the code
            $books = Books::with('autor')
            ->with('reviews')->where(['author_id' => $authorId])
            ->get();

            foreach($books as $book) {
                // some business logic here...
                // this code snippet can turn out to be huge,
                // since it grows with the application complexity.
            }

            return view('home.home')->with(['books' => $books]);
        } catch (Exception $e) {
            return back()->with('error', $e->getMessage());
        }
    }
}


[Ollie] Gee, you are right! I want my applications to be able to scale! How can I separate my code into more layers?

[Stan] I thought so. Essentially, a Controller’s job is to interact with the View (the V from MVC) layer. This means that it should handle the user input, and serving back the data to be displayed to the user.

Nothing more, nothing less.

Who is responsible for each layer?

Having said that, let’s revise the layers and see which of them should remain in the Controller and which should be broken apart:

  • Validation Layer
    This layer is responsible for validating data entered by the user. In the MVC diagram, it exists very close to the View layer, so it should be implemented in the Controller class.

  • Business Logic Layer
    This layer is usually the one that gets too complicated over time. It defines the Business rules of the application and has nothing to do with the View. So, we need to decouple it from the Controller and package it into another Class in the Business Logic Layer.

  • Database/ Repository Layer
    This layer includes the DB queries of the application. In many complex applications that are data intensive (such as real-time systems), this layer may also be a different application by itself. So it should not be implemented in the Controller, but in another Class living in the Database/ Repository Layer.

  • Error Handling Layer
    What should we do when an Exception is thrown? It depends. Maybe we want to Log the Exception into a logging channel and take a special action.
    In most MVC applications we want to inform the User about the error, so this layer should be implemented both in the Business Logic layer and the Controller layer.

  • Success/ Error displaying layer
    This layer is coupled with the previous one. When an operation is successful, or when an Exception has been thrown, it is of big importance to inform the User accordingly. This layer is defined between the Controller and the View, and can be implemented in the Controller Class.

[Ollie] Wow, I learnt so much! But I am still a bit confused; How should my Controller look like now?

Layered Code

[Stan] Cool question! Look at the following example:


<?php
namespace App\Http\Controllers;

class BooksController extends Controller {

  protected $bookManager;

  function __construct() {
      $this->bookManager = new BookManager();
  }

  public function complexMethod(HttpRequest $request) {
      // VALIDATION LAYER
      // having all rules in a separate validationRules method
      // allows reusage
      $validator = Validator::make($request->all(), $this->validationRules($request));

      if ($validator->fails()) {
          return back()->with('error','Wrong data');
      }

      try {
          // BUSINESS LOGIC LAYER
          $books = $this->bookManager->getAllBooksForAuthor($request->author_id);

          // SUCCESS DISPLAY LAYER
          return view('home.home')->with(['books' => $books]);
      } catch (Exception $e) {
            //ERROR DISPLAY LAYER
        return back()->with('error', $e->getMessage());
      }

  }

}

[Ollie] But where are the Business Logic methods and the Database/ Repository layer methods?

[Stan] Defined in other classes, of course! One of the best things about Object Oriented Programming is the ability to package different modules into separate classes and use them by creating instances of those classes.
(See the constructor in the last example).

Let's see our BookManager class:


<?php  
namespace App\BusinessLogicLayer\;
use App\StorageLayer\BookRepository;

class BookManager {

    protected $bookRepository;
    const PUBLISHED_BOOK_STATE = 1;

    public function __construct() {
        $this->bookRepository = new BookRepository();
    }

    public function getAllBooksForAuthor($authorId) {
        // here we can add all the business logic:
        // for example, we can check whether the author has any 
        // books that are in a DRAFT state, or we can check if the
        // author is the same user as the logged in user, in order
        // to display more data.

        $books = $this->booksRepository->getAllBooks([
            'state' => self::PUBLISHED_BOOK_STATE,
            'author_id' => $authorId
        ]);   

        foreach($books as $book) {
            // business logic here
        }

        return $books;
    }
}

[Ollie] But I don't see any DB queries here, either.

[Stan] Exactly! The DB queries are in Repository/ Storage layer, remember? Look at the following class:


<?php 

namespace app\StorageLayer;
use app\models\Book;

class BookRepository {

    public function getAllBooks($attributesArray) {  
        return Book::where(attributesArray);  
    }

}

Thinking ahead

[Ollie] Isn't this class very small? Can't we just omit it and include the DB query to the Business Logic Layer?

[Stan] Not if we want to scale correctly. Remember, as your project grows, there might be a need to have complex DB queries in our class or even transfer the Repository layer in a totally separate project (even in a different server).

[Ollie] Wow, I really haven't thought of all these! You are right. This code looks clean and scalable. I guess it makes adding more functionality way easier.

[Stan] Exactly. In complex applications, it is essential to follow the Open-closed principle:

"software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification"

So, even if you have written a Controller method a long time ago and want to write another one that reuses some of the layers defined in other classes, you can simply call the relevant methods in the new Controller method.

[Ollie] That is so cool! I feel ready to scale now!

Scale Projects Meme

enter image description here

Posted on by:

pavlosisaris profile

Paul Isaris

@pavlosisaris

Software Engineer @SciFY. Live to learn something new -and write cleaner and more sustainable code- every day. Passionate with learning and discovering new technologies, history, and psychology.

Discussion

markdown guide
 

Isnt MVC now something considered to be a first step while true scalability is with Redux like models? There was a facebook conference video that very clearly demonstrates how mvc alone was still failing facebook to scale and the redux model lead to creation of react?

UPDATE: here is the video: youtube.com/watch?v=nYkdrAPrdcw

 

Can you share with us the video, Sergio? Looks interesting.
Indeed, MVC can be limiting if you contraint your code only in the Controller.
But by mdularizing your code in other classes helps when designing to scale.

 

Yeah I would agree that MVC is a great concept and then I think even more granular - modularization of C + putting some thinking into "source of truth" for data and overall "state management" ensuring there is logic and order.
I think I did 3 major refactors to achieve that.

but I also would say - when I am prototyping (coding fast and dirty) I am actually fine with writing non scalable code as I don't see a problem to refactor it later on.

 
 

All layers in your example are useless because of ORM models(persistent layer) leaking through repository and business logic right into the controller.

If you want to go with layers then you need business level entities to be separated from persistence level. This can be done with data mapper.

There is no point in repository that returns ORM models to you

 

Thanks for your comment Dmitry. ORM is indeed very handy (and Laravel's Eloquent is very powerful), but you need to know how to use it, or which of it's features will obstruct your app's scalability.

 

My point is that if you go with layers then all ORM stuff should end up in persistence layer. Your controllers and business logic should know nothing about ORM

Hmm I get what you are saying. Do you mean that I shouldn't refer to DB table fields (like author_id) in the business logic layer?

More than that. You should not even have access to the database on the business logic layer. It should be done in persistence. With your current approach you pass ORM models into the view level where everyone will have direct access to the database because of access to ORM model. Somebody can make something like $model->delete() in the view template and mess the things up.

To prevent it you need to decouple persistence level and all other levels. In the persistence level itself you are free to use Eloquent, Doctrine, raw SQL queries, etc...

For more information please refer to fideloper.com/how-we-code

 

I just spotted the private keyword in your code, which triggered a need to rant.

Personally I really hate code that uses the private keyword even once. It's pretty much always the wrong solution, unless you're delivering code to 100% externals who are intended to have limited access points. At least issues with protected code can be fixed (in most cases) relatively easily by extending the class.

In frameworks, or your own code that is supposed to be used by your own people, it's always wrong.

I can give several examples of issues I've had that are purely due to someone thinking they're smarter than every other programmer and thus using private to lock down their code from modifications (the only reason I've ever figured out for using it).

Just a few that first come into mind:

  • OAuth library, which we needed to reconfigure in automated tests just slightly, and ended up having to use reflection to change a private method to public
  • Frameworks have had bugs that would be easy to work around if the methods were protected or public so I could e.g. manually clear some state, even protected would've allowed extension to fix, but instead had to copy & paste the whole class and change one word from private to public to work around the issue
  • Database layer where someone built an optimization which caches lots of things internally, but in certain cases the cache caused problems and out of control memory use when we tried to batch process a few million entries in the db - it had a method for clearing the cache, which was private

Code should optimally be written with exactly zero attempts to limit what other developers can do with it, but hints as to what they probably shouldn't be using if they don't know what they're doing. The _ prefix for functions and variables, as e.g. Python has standardized, tends to be good enough.

This way, when you're trying to interface with something, you see the methods you're supposed to be calling for normal operation, and when you run into problems with your specific use case you can work around that, and then open an issue to get the issue solved later on.

 

Hey Janne, thaks for your fruitful comment. You are right, I also tend to use protected instead of private in my code, so that I can be able to extend the class and create sub-classes.

However, the point of the code snippets in this article is to provide a view of a layered MVC architecture and modularized classes.

In a more realistic scenario one would have preferred to use protected, for all the reasons you referenced.

 

I dont want to be "that guy" but... usually when we talk about scale-ability, its in terms of load balancing etc.

What you're referencing is maintainability, right? this is something that would confuse a lot of newcomers.

I could be wrong, I'm open to hear how if so.

If its not maintainability, then how does refactoring blocks of code make your project more scalable?

It makes it more readable and ofcourse, better to maintain when we go past a certain stage and our project is no longer a baby.

However, it doesn't reduce database load, improve the amount of connections our application can handle and so on.

 

However, just to provide some code examples in Laravel.

An unscalable chunk of code(in my opinion) would be this:


$users = User::get();

foreach($users as $user) {
    $user->name = 'Bob'; // We set them all the same name for example
    $user->save(); //This would be a query for each user
}

If we had 100 users, this would be 101 queries.


//Instead, for this example, we could just call this:

User::update(['name' => 'bob']); 

Resulting in just one query.

 

Hey Sam,
I agree with your point of view. By "Scaling", I essentially mean to prepare the app not only for more users and DB rows, but for new client requirements and Business Logic.
Sure load balancing will be a nice solution If you plan to make your 100 usres to be 10000. But will not help you much with new business requirements.
So yes, by "Scaling" I do mean "Easy to maintain and grow in terms of business requirements".

 

I've found the Hierarchical MVC pattern a good one for scalability: en.wikipedia.org/wiki/Hierarchical...

 

Thanks for this - haven't heard it before. Isn't it similar with the MVVM (Model - View - ViewModel)? en.wikipedia.org/wiki/Model%E2%80%...

 

Hmmm not that I can tell. I think H-MVC is more like a combo between Chain-of-Responsibility and MVC. In any case, it came in handy with a complex Java/Swing GUI I had to build way back, where my first attempt was to squeeze the whole thing into a single MVC layer, which ended up way too complex and hard to maintain.

 

You really explained it so well! .I have learnt a lot in ten minutes! Thank you.

 
 

What a great post. The way you explained how to separate each layer was concise and clear. Thanks for sharing!

 

Thanks for sharing your opinion on my post, Martin! :)

 

Shouldn't

} catch (Exception $e)

be

} catch (\Exception $e)

?

Relating the article, I agree, though I just leave in the Manager real complex shared logic, I've all the rest in the repo class.

 

You are right! However, the puprose of the code snippets are not to be syntactiaclly correct, but to give an overview of the architecture.

 

Good explanation of why we should follow SOLID Rules