DEV Community

Cover image for Template::Stencil - A Fast Template Engine
LNATION
LNATION

Posted on

Template::Stencil - A Fast Template Engine

#perl #programming

A template engine spends its life doing the same job over and over. The task does not change between request one and request one million. The variables do. Yet most engines re-derive some part of the page on every render: they walk a tree of node objects, they build and throw away intermediate strings and they allocate a fresh buffer per fragment.

Template::Stencil takes the other route. A template is parsed once into packed bytecode in a single memory arena, and rendered by a threaded C interpreter that writes straight into an SV-backed buffer. The scalar you get back is that buffer. At steady state a render performs no heap allocation and no system calls at all.

use Template::Stencil;

my $stencil = Template::Stencil->new(
    template_dir => 'templates',
    wrapper      => 'wrapper.tmpl',
);

my $html = $stencil->render('index', {
    title => 'Hello',
    items => [ { name => 'one' }, { name => 'two' } ],
});
Enter fullscreen mode Exit fullscreen mode 
<h1>{% title | upper %}</h1>
<ul>
{% for item in items %}<li>{% item.name %}</li>
{% end %}</ul>
Enter fullscreen mode Exit fullscreen mode

One delimiter, one closer

Everything is {% %}. Every block closes with {% end %}. Whitespace inside the delimiters is insignificant, so {%name%} and {% name %} are the same tag.

Output tags 

{% name %}              variable, HTML-escaped
{% page.header %}       dotted path through hashrefs
{% page.number[0] %}    array index; mixes freely: a.b[0][1].c
{% raw name %}          same resolution, no escaping
{% name | upper %}      filters
Enter fullscreen mode Exit fullscreen mode

Auto-escaping is on and replaces < > & " ' with entities. raw is the opt-out. A missing or undefined path renders as the empty string, or croaks if you asked for strict.

Paths resolve innermost scope first: loop variables and set bindings shadow outer scopes, and the hashref you pass to render is the outermost scope. Paths go up to eight segments deep, and traversing through a blessed reference is an error - templates render data, they do not call methods.

Filters 

{% name | upper %}
{% price | default('0.00') %}
{% summary | trim | lower %}
Enter fullscreen mode Exit fullscreen mode

Filters chain left to right. Escaping happens once, after the last filter, unless the tag is raw or the chain ends in html. Nothing is ever double-escaped, though a filter that runs after html re-escapes, because it changed the bytes.

The built-ins are all C:

filter does
upper uppercase (ASCII fast path, UTF-8 aware)
lower lowercase
trim strip leading and trailing whitespace
html HTML-escape now, and mark the value escaped
uri RFC 3986 percent-encoding of the component
default(x) replace undef or '' with the literal x
fmt(f) sprintf with exactly one conversion

fmt is the number formatter:

{% price | fmt('$%.2f') %}      $1.50
{% id    | fmt('%08d')  %}      00000042
{% hash  | fmt('%x')    %}      deadbeef
Enter fullscreen mode Exit fullscreen mode

The format is one sprintf conversion from diouxXeEfgGs, with the usual flags, width and precision, surrounded by any literal text. It is validated when the template compiles, so a second conversion, a *, a length modifier, %n or an oversized width is a compile-time error rather than a render-time surprise. Integer conversions cover the full IV/UV range on every platform, and %s with a precision counts bytes without leaving a broken UTF-8 sequence behind.

Your own filters are coderefs registered on the constructor:

filters => { money => sub { sprintf '%.2f', $_[0] } }
Enter fullscreen mode Exit fullscreen mode 
{% price | money %}
{% s | repeat(3) %}     # coderef called as ->($value, 3)
Enter fullscreen mode Exit fullscreen mode

An argument is a single string or number literal. An unknown filter name is a compile-time error that lists what is registered, and a die inside a filter becomes a render error carrying the filter name and the template location.

User filters are the escape hatch, not the fast path. Crossing the Perl call boundary is roughly 100ns before the body of your sub runs, where a built-in is free. If the coderef is only a sprintf, use fmt.

Conditionals 

{% if expr %} ... {% elsif expr %} ... {% else %} ... {% end %}
{% unless expr %} ... {% end %}
Enter fullscreen mode Exit fullscreen mode

The expression grammar is deliberately small - no arithmetic, no concatenation:

operands     paths, numbers (42, 3.14, -2), strings ('a' or "a"), undef
numeric      ==  !=  <  >  <=  >=
string       eq  ne  lt  gt  le  ge
boolean      && / and,  || / or,  ! / not,  ( ... )
other        defined(path)
Enter fullscreen mode Exit fullscreen mode

&& and || short-circuit and yield the deciding operand, exactly like Perl. Comparison operators are typed at compile time from how they are spelled: == compares numerically, eq compares strings, and the VM never has to guess.

Truthiness follows Perl - undef, 0, '' and '0' are false - with one deliberate extension: an unblessed empty arrayref or hashref is false too, so {% if items %} guards a loop the way you would expect it to.

Loops 

{% for item in items %} ... {% end %}
{% for key, value in hash %} ... {% end %}
Enter fullscreen mode Exit fullscreen mode

Arrays bind each element. Hashes bind key and value and iterate in sorted key order by default, so output is deterministic; sort_keys => 0 gives you raw hash order. Iterating undef or an empty aggregate renders nothing.

Inside a loop there is an implicit loop variable:

loop.index    0-based index          loop.first   true on first
loop.index1   1-based index          loop.last    true on last
loop.size     total iterations       loop.even    parity of index1
loop.key      current key (hash)     loop.odd     parity of index1
Enter fullscreen mode Exit fullscreen mode

loop always means the innermost loop. To reach an outer one, capture it:

{% for item in items %}
  {% set item_loop = loop %}
  {% for x in item.list %}
    {% item_loop.index %} / {% loop.index %}
  {% end %}
{% end %}
Enter fullscreen mode Exit fullscreen mode

Assignment 

{% set name = expr %}
Enter fullscreen mode Exit fullscreen mode

The value is any expression from the grammar above, and the binding lives in the current block scope. A set inside a for body is fresh each iteration and gone after the {% end %}; a set inside an if branch dies with the branch; a top-level set lasts to the end of the template. Bindings shadow your data without modifying it - the hashref you passed to render is never touched.

Includes 

{% include header.tmpl %}
{% include header %}        # .tmpl appended when the name has no dot
Enter fullscreen mode Exit fullscreen mode

The name is static and resolves against template_dir. An include shares the current scope, so it sees loop variables, set bindings and the root data exactly as the include site does. Includes are compiled once and linked, and revalidate independently: editing an include takes effect without recompiling everything that includes it. Cycles are a compile-time error naming the cycle, and absolute paths and .. segments are refused.

Wrapper 

my $stencil = Template::Stencil->new(wrapper => 'wrapper.tmpl');
Enter fullscreen mode Exit fullscreen mode 
<html><body>{% content %}</body></html>
Enter fullscreen mode Exit fullscreen mode

With a wrapper configured, render runs the wrapper and {% content %} renders the requested template at that point. This is one pass into one buffer, with no intermediate string for the inner page. The wrapper sees the same data. Override per render with { wrapper => 'other.tmpl' } or disable it with { wrapper => undef }.

A wrapper without a {% content %} is refused at compile time, rendering a template that contains {% content %} directly is a render error, and {% content %} runs exactly once.

Comments and literal braces 

{%# anything, up to the first closing percent-brace %}
Enter fullscreen mode Exit fullscreen mode

Comments are stripped at compile time and may span lines. A literal {% in output is written {%%}, the empty tag. A bare %} or } in text needs no escaping at all.

The Perl API

new 

my $stencil = Template::Stencil->new(%options);
my $stencil = Template::Stencil->new(\%options);
Enter fullscreen mode Exit fullscreen mode

Unknown option names croak, which is the theme: things that are knowable when you build the engine are checked when you build the engine.

option default meaning
template_dir none base directory for file templates and includes
wrapper none default layout template
filters {} user filter registry, every value a coderef
auto_escape 1 HTML-escape output tags
strict 0 croak on an undef or missing value, with the path and location
cache 1 cache compiled templates
cache_size 256 bound on cached string-keyed templates, LRU evicted
stat_ttl 1 seconds between mtime revalidations of cached files
sort_keys 1 deterministic sorted hash iteration
chars 0 return a character string instead of UTF-8 bytes
pretty 0 re-indent the rendered HTML through Eshu

stat_ttl is the production knob. 0 stats on every render, the default 1 stats at most once a second per template, and a negative value never re-checks at all, which is what gets you to zero syscalls at steady state. File templates are cached with mtime revalidation; string templates are cached by content hash in the bounded LRU.

pretty loads Eshu on first use. It is an optional dependency: asking for pretty without it is an error, and never asking for it costs nothing. Budget around 1.5 microseconds per KB when you do.

render 

my $out = $stencil->render($template, \%data);
my $out = $stencil->render($template, \%data, \%opts);
Enter fullscreen mode Exit fullscreen mode

$template is either source or a file name. The rule is mechanical: an argument with no newline and no {% that resolves to a file - under template_dir, or relative to cwd, with .tmpl inferred - renders that file, and anything else is treated as source. Rendering equal content as a string and as a file produces byte-identical output.

\%data is the root scope and is optional; anything that is not a hashref croaks. \%opts overrides wrapper, strict and pretty for this one render, and unknown keys croak.

The return value is the render buffer itself, a fresh scalar each call, which is exactly what you want to hand to a PSGI body arrayref.

One detail worth knowing: a statically resolved call, written Template::Stencil::render($s, ...), is rewritten at compile time by a call checker onto a direct C entry point, skipping the usual XSUB dispatch. Method calls take the ordinary path. Both produce identical results, and the benchmark below uses the static form because a hot loop in a real application would.

Template::Stencil::PSGI

The small conveniences that everyone writes anyway:

use Template::Stencil::PSGI;

my $view = Template::Stencil::PSGI->new(
    template_dir => 'templates',
    wrapper      => 'wrapper.tmpl',
);

# a finished PSGI response tuple, Content-Length included
my $res = $view->res('index', { title => 'Hi' });

# or a tiny routed app
my $app = $view->to_app({
    '/'      => [ 'index', sub { my $env = shift; { title => 'Hi' } } ],
    '/about' => 'about',
});
Enter fullscreen mode Exit fullscreen mode

new takes exactly the Template::Stencil options, stencil hands back the underlying object, res($template, \%data, $status, \@headers) builds the tuple, and to_app(\%routes) gives you an app where a route value is either a template name or [$template, \%data | sub ($env) { ... }]. Unknown paths get a 404.

Encoding, errors, concurrency

Encoding is the engine's job. Callers never utf8::encode. By default render returns wire-ready UTF-8 bytes: UTF8-flagged values pass through, unflagged latin-1 values and templates containing high bytes are upgraded automatically, and uri percent-encodes UTF-8 bytes. length($out) is therefore the correct Content-Length, which is the entire point. Template files are expected to be UTF-8 or ASCII on disk. For non-web use, chars => 1 returns a flagged character string instead.

Errors croak with the template name, a line, a column for compile errors, and a message:

Template::Stencil: templates/page.tmpl:12:5: unclosed 'if' block
Template::Stencil: header.tmpl:2: undef value for 'user.name'
Enter fullscreen mode Exit fullscreen mode

An error inside an include or a wrapper names the failing file, not the page that pulled it in. Unclosed blocks report the opener's position, and mismatched constructs name both ends.

Concurrency is one engine per interpreter. Nothing is shared, so nothing is locked, and there are no locks anywhere in the codebase. A preforking server constructs the object before or after fork and each child gets a private copy either way. Under ithreads a cloned object lazily rebuilds its own engine, with an empty cache, from its cloned options on first use.

What it will not do

The honest list.

  • Blessed references cannot be traversed in paths. No method calls in templates; the engine renders data.
  • Empty unblessed aggregates are false in conditions. This is a documented extension to Perl truthiness, not an accident.
  • Hash-loop key order is sorted bytewise by default.
  • The reserved words if unless elsif else end for set include raw content in cannot be used in the first position of a tag, or as a for or set binding name. They work fine as data keys.
  • Dynamic include names, wrapper chains, arithmetic in expressions, context-aware escaping for attributes/JS/URLs, i18n and streaming render are all reserved hook points, not shipped features.

The numbers

The distribution ships bench/compare.pl, which renders the same reference page in every engine it can find installed:

perl -Mblib bench/compare.pl
Enter fullscreen mode Exit fullscreen mode

The page is about 1 KB - a title, a heading with characters that actually need escaping, a conditional, a paragraph of filler and a ten-item loop over hashrefs. Every engine gets its own idiomatic spelling of that page, with caching turned on and warmed, and HTML escaping on where the engine makes it optional. On my machine:

engine ops/sec ns/op relative
Template::Stencil 1,442,829 693 1.00x
Text::Xslate 559,685 1,787 2.58x
hand-written perl 190,015 5,263 7.59x
Template::Toolkit 42,658 23,442 33.82x
Text::Template 35,213 28,398 40.97x
HTML::Template 22,984 43,509 62.78x

Where it is

Template::Stencil is on CPAN. It needs Perl 5.10.1 and a C compiler, builds on Linux, macOS, the BSDs and Windows (MSVC included), and has no runtime dependencies at all. Eshu is optional and only for pretty.

It is young. The syntax is settled and I do not intend to grow it much - the small grammar is the feature - but the edges are not. If you try it and something is wrong or missing, I would like to hear about it.

Top comments (0)