Catalyst Tricks: Map Request Parameters to a Model

Introduction

Dealing with incoming request parameters (both query and body parameters) is something nearly all Perl Catalyst applications need to cope with. Unfortunately Catalyst punts here and doesn't give you a lot of guidance and the built in handling leaves a lot to be desired. In this blog I will first example how the default handling works, some of the problems with it and how Catalyst developers have tried to improve it over the years (with minor success IMHO; I can say that since half the redos are my fault ;)).

How Catalyst Handles Request Bodies and Query Parameters

By default incoming query and body parameters get mapped to the Catalyst Request object:

  $c->request->query_parameters
  $c->request->body_parameters

query_parameters gives you access to parameters passed in the 'query' section of your request URL. For example if your URL is https://example.com/page/?aaa=1&bbb=2 then query_parameters will return the following hashref:

  +{
    aaa => "1",
    bbb => "2",
  }

The body_parameters method gives you access to classic HTML Form POST bodies. For example if you have an HTML Form like this:

<form action="/login" method="post">
    <label for="username">Username:</label>
    <input type="text" id="username" name="username">
    <label for="password">Password:</label>
    <input type="password" id="password" name="password">
    <button type="submit">Login</button>
</form>

When the user clicks the submit button you would expect the following hashref in body_parameters:

  +{
    username => "$USERNAME",
    password => "$PASSWORD",
  }

(Substitute $USERNAME and $PASSWORD for whatever the user typed into the form).

Both method return a hashref of key value pairs where the key in the field or parameter and the value is a scalar or arrayref (depending on if there is one or several values for the given field in the request).

For basic applications this has worked acceptably but there's a number of issues. First of all the fact that the key can be either a scalar or arrayref is annoying, requiring you to write tons of defensive code like:

my $username = $c->req->body_parameters->{username};
$username = ref $username eq 'ARRAY' ? $username[-1] : ($username);

Or just ignore the problem and potentially open yourself to security issues. Speaking of security issues I don't know how many times I've seen code like this, passing incoming body parameters straight into a DBIx::Class object:

  my $new_user = $c->model('Schema::User')->create($c->req->body_parameters);

This is a world of hurt since you are basically passing whatever the user submitted (or your site hacker is submitting) directly to DBIC create. You need to be more choosey about the incoming at the very least:

  my $new_user = $c->model('Schema::User')->create(
    username => ref($c->req->body_params->{username}) eq 'ARRAY' ? $c->req->body_params->{username}[-1] : $c->req->body_params->{username},
    password => ref($c->req->body_params->{password}) eq 'ARRAY' ? $c->req->body_params->{password}[-1] : $c->req->body_params->{password},
  );

At which point you are starting to have a lot of ugly code and you haven't even started on form validation yet. And with all this repeated code its easy to have a hard to spot typo:

  my $new_user = $c->model('Schema::User')->create(
    username => ref($c->req->body_params->{username}) eq 'ARRAY' ? $c->req->body_params->{usrname}[-1] : $c->req->body_params->{usernme},
  ...

I've seen a lot of typo issues in Catalyst applications just like this, and they can lead to hard to spot problems since in Perl having a typo in the hash de-reference will not lead to a hard runtime error generally, you just get 'undef' for a value in an unexpected location. I've seen this problem in Catalyst code which existed for years in the wild.

Another thing I've seen a lot of is line after line of parameter processing code stuck into controllers. As it turns out parameter munging is one of the bigger jobs a programmer in a web application can have, especially as the application gets older and you need to introduce new features without breaking backward compatibility. This can lead to very long and ugly controllers that make following the flow of logic in your request to response cycle difficult.

You can solve the 'is it a value or an arrayref?' problem by enabled the use_hash_multivalue_in_request configuration option. This gives you a Hash::MultiValue object instead of a hashref of request parameters. Amongst other things it make it easy to say 'when there's more than one value give me only the last one', which is nearly always the right thing as legitimate uses for this typically revolve around HTML Form tricks where some field types like checkboxes don't make it easy to know when the user is explicitly setting an 'off' state. See CONFIGURATION For more. This however doesn't really help with the other problems such as easy to make typos or dirtying up your controllers with tons of parameter tweaking code.

Mapping Incoming Request Parameters to a Model

One trick I've used for years when encountering this issue is to use a Catalyst Model as a container for my request parameters. This model converts the hashref to an actual object with methods, which means any typos get picked up at runtime fast. This model is also a great place to stick validations and incoming value filters, as well as a good spot to stick and complex logic involving these parameters. Let's keep this simple and just see how one might do that for the example login form already described:

package Example::Model::Params::Login;

use Moose;
use Valiant::Validations;
use Valiant::Filters;

extends 'Catalyst::Model';
with 'Catalyst::Component::InstancePerContext';

sub build_per_context_instance {
  my ($self, $c) = @_;
  my $body = ref($self)->new(%{$c->req->query_parameters}, ctx=>$c);
  return $body->validate; # ->validate returns '$self' for chaining
}

has ctx => (is=>'ro');

has username => (
  is => 'ro',
  validates => [
    presence => 1,
    length => {
      maximum => 64,
      minimum => 1,
    },
  ],
);

has password => (
  is => 'ro',
  validates => [
    presence => 1,
    length => {
      maximum => 64,
      minimum => 1,
    },
  ],
);

has user => (
  is => 'ro',
  lazy => 1,
  predicate => 'has_user',
  default => \&_find_user,
  validates => [
    presence => {message=>'User Not Found with credentials.'},
  ],
)

filters_with 'Truncate', max_length=>100;

sub _find_user {
  my $self = shift;
  my $user = $self->ctx
    ->model('Schema::User')
    ->find({username=>$self->username});

  return unless $user && $user->password_eq($self->password);  
  return $user;
}

This cleanly encapsulates the entire job of getting the POSTed parameters, making sure they are valid and that the parameters match a user in the database (and that the given password matches the latest in the DB via the password_eq method, which is an exercise I leave for you; don't forget to hash your passwords in the DB!).

You can use it in a controller similar to:

package Example::Controller::Session;

use Moose;
use MooseX::Attributes;

extends 'Catalyst::Controller';

sub login :Path Args(0) {
  my ($self, $c) = @_;
  my $params = $c->model('Params::Login');
  return $c->login_user($params->user) if $params->valid;
  return $c->stash(params => $params);
}

In this example we map the incoming request body to $params and if the object is valid we perform the login workflow (via login_user($user), a method that again I leave to your imagination but probably involves storing the user id in the session and redirecting to some sort of "You're Logged in" page). If it not we stick $params in the stash and let the view inspect it for the errors, displaying such to the user in whatever view system you prefer.

Conclusion

In real life you'd probably use the Authentication plugin for something like this, but the general idea here can map to basically any type of incoming query or request bodies, even those via APIs requests that might be in JSON rather than a form POST. What you get is a nice, clear separation of concerns that improves code readability and long term maintainability. I like the idea so much, and started using it so much that I've tried to encapsulate the pattern in CatalystX::RequestModel. Other approaches on CPAN that can do similar would be HTML::FormHandler, although that tends to focus more on validation and HTML Form field generation, so might be a bigger hammer than you want.

Modules mentioned in this blog include Catalyst and Valiant

Bonus Idea

I often use a similar approach to wrap the Catalyst session (also represented as a hash reference) in a model, to offer a strongly typed interface to the session. Can you figure out the code for that?