Context::Vars is another new CPAN distribution I've published which solves a familiar problem in Perl programs. Some values belong to one request or one job: a request id, the current user, a tracing span. Code far down the call stack needs to read them, and so do callbacks that run later. Sometimes they need to be overriden within specific context.
The usual answers each fail in a different way. A global gets overwritten as soon as two requests are in flight on one event loop. local is undone when the sub returns, which is before a callback registered in it ever runs. Passing a $c everywhere works, until the logger, the database layer or the tracing library you did not write needs it too.
Context::Vars gives you a variable that holds a value per context, and the context follows the work: into every sub called, into callbacks, and across await.
use Context::Vars qw($request_id);
Context::Vars
A variable, imported
Every package that imports a name shares the one variable. What it reads is the value in the context current at that moment:
package My::Log;
use Context::Vars qw($request_id);
sub info { printf "[%s] %s\n", $request_id // '-', shift }
package My::App;
use Context::Vars qw($request_id);
sub handle {
my ($req) = @_;
$request_id = $req->{id};
My::Log::info('start');
load_user($req);
My::Log::info('done');
}
sub load_user { My::Log::info('loading user') }
handle({ id => 'a1' });
My::Log::info('between requests');
handle({ id => 'b2' });
[a1] start
[a1] loading user
[a1] done
[-] between requests
[b2] start
[b2] loading user
[b2] done
The logger never takes an argument for the request. Between requests it reads nothing, because handle's assignment ended when handle returned.
The rule: an assignment belongs to the sub it is written in
use Context::Vars q{$variable};
$variable = 'one';
sub change {
$variable = 'two';
print "$variable\n"; # two
}
change();
print "$variable\n"; # one
An assignment holds until the sub it is written in returns, however it returns: falling off the end, return from inside a loop, die, goto &other, last out of the caller's loop. A block inside the sub does not end it. Every sub it calls sees it. Code outside any sub keeps what it assigns for the program.
It is like local, with two differences that matter: you do not have to write it, and it survives into callbacks.
Callbacks see the values of the moment they were wrapped
An event loop, a queue or a future runs a callback later, after other work has changed things. Wrap the callback, and it runs with the values of the moment it was wrapped:
my @queue;
sub later { push @queue, Context::Vars->wrap(shift) }
sub handle {
my ($id) = @_;
$request_id = $id;
later(sub { My::Log::info('callback ran') });
}
handle('a1');
handle('b2');
$_->() for reverse @queue;
[b2] callback ran
[a1] callback ran
Without wrap, the same two callbacks print [-] callback ran twice. They run outside any request, and they never see another request's value instead of their own. Nothing is the worst answer, but it is never a wrong one.
async subs
Under Future::AsyncAwait, an async sub keeps its own values across await, however the waits interleave:
use Future::AsyncAwait;
my %pending;
async sub handle {
my ($id) = @_;
$request_id = $id;
My::Log::info('waiting');
await($pending{$id} = Future->new);
My::Log::info('resumed');
}
my @f = (handle('a1'), handle('b2'));
$pending{b2}->done;
$pending{a1}->done;
[a1] waiting
[b2] waiting
[b2] resumed
[a1] resumed
Context::Vars does not depend on Future::AsyncAwait. If it is loaded, before or after, Context::Vars hooks into its suspend and resume.
Holding a context yourself
capture keeps the values every variable has now. run runs code with them:
use Context::Vars qw($user);
my $ctx;
sub login {
$user = 'alice';
$ctx = Context::Vars->capture;
$user = 'alice, later';
}
login();
$ctx->get('user'); # alice
$ctx->names; # user
$ctx->run(sub { "hello $user" }); # hello alice
$user; # undef: login has returned
local works too
use Context::Vars qw($level);
$level = 'info';
sub show { print "$level\n" }
{
local $level = 'debug';
show(); # debug
}
show(); # info
Arrays, hashes and whole structures
Arrays and hashes are imported the same way and follow the same rule. A push, an element stored or deleted, a whole assignment: each holds until the sub it is written in returns.
use Context::Vars qw(@trail %seen);
sub step {
my ($name) = @_;
push @trail, $name;
$seen{$name}++;
return "@trail";
}
sub handle {
push @trail, 'handle';
print step('validate'), "\n"; # handle validate
print "@trail\n"; # handle
}
The rule goes all the way down. A plain hash or array stored in a context variable, at any depth, belongs to the context:
use Context::Vars qw(%seen $config);
%seen = (user => { roles => ['reader'] });
sub elevate {
push @{ $seen{user}{roles} }, 'admin';
print join(',', @{ $seen{user}{roles} }), "\n"; # reader,admin
}
elevate();
print join(',', @{ $seen{user}{roles} }), "\n"; # reader
$config = { level => 'info', db => { host => 'localhost' } };
sub tweak { $config->{db}{host} = 'replica' }
tweak();
print "$config->{db}{host}\n"; # localhost
Objects and code references stored anywhere in such a structure are kept as the references they are, and shared, as their classes expect.
One thing that surprises people
A sub that only assigns a variable for its caller changes nothing the caller can see:
sub set_user { $user = shift }
sub handle {
set_user('alice');
print $user // 'undef', "\n"; # undef
$user = lookup_user(); # do this instead
}
The assignment belongs to set_user, and set_user has returned. Return the value and let the caller assign it.
How it works
1. A context is a persistent map
A context is a map from variable name to value, and it is a HAMT::Map from the HAMT distribution: a persistent map, where an update returns a new map and leaves the old one exactly as it was, sharing almost all of its memory with it.
That is what makes everything else cheap. Keeping a context for a callback is copying one small struct onto the same tree. A sub that assigns gets a fresh copy, and its store copies the four or five nodes on one path. Every other context still points at the nodes it had.
There is one current context for each interpreter. Code at the top level, before anything has been assigned, runs in a shared empty one that is never written to.
2. The variable is a scalar with magic
use Context::Vars qw($request_id) puts one scalar into your package, the same scalar in every package that imports the name. It holds no value of its own. Reading it fetches request_id from the current context; assigning it stores into the current context. use strict accepts it as it accepts a name from use vars.
3. The scope is decided when the sub is compiled
"The sub it is written in" is lexical, so it is settled at compile time. When perl finishes compiling a sub it hands it to its optimiser, and Context::Vars hooks that. The hook walks the sub's op tree looking for a write to one of its variables: an assignment, .=, ++, chomp, push, an element store at any depth, local.
A sub that has one is given a scope. Its first statement is changed to open one, every time the sub is entered: a fresh copy of the current context becomes current, and putting the previous one back goes onto perl's save stack. That is the same stack local uses. However the sub is left, by return, die, goto or anything else, the save stack is unwound and the caller's context comes back.
A sub that only reads is left exactly as perl built it, and costs nothing extra to call.
This runs the same on every perl from 5.10 to 5.44. Getting there meant learning, among other things, that perl 5.14 and older optimise a map block as soon as they parse it, and that @{ $h{list} } is really a little block whose value is its last statement.
4. A callback carries its context
wrap captures the current context, one struct copy, and returns a sub that, each time it is called, makes a fresh copy of that capture current, calls your code, and restores whatever was current before. Because each call starts from a fresh copy, nothing one call does is seen by the next.
5. async subs
An async sub leaves by an ordinary return when it suspends at await, so its scope closes, which is right. Nothing reopens it when it is resumed, which would be wrong. Future::AsyncAwait has hooks that run around every suspend and resume. Context::Vars uses them to keep what the sub could see when it first suspended, and to make that current again for each stretch it runs.
6. Arrays and hashes are proxies over persistent data
A plain hash in a context is held as a HAMT map, and a plain array as a HAMT vector, at every level. What you read back is a hash or array tied to a path: the variable's name, then each key or index. Each access resolves the path against the current context. A change at any depth copies the nodes on the path from the variable down to the element, and stores the new top under the variable's name, in the current context.
The visible difference from an ordinary hash is identity: two reads of $seen{a} are two references, and == between them is false. Copy one out, { %{ $seen{a} } }, when you want an ordinary hash.
A C ABI for event loops
An event loop written in C can capture a context when a callback is registered and enter it when the callback runs, with no Perl call in between. The interface is one installed header, context_vars_abi.h, found through ExtUtils::Depends, and a loop needs three calls per callback:
cb->ctx = (VARS->capture)(aTHX); /* when it is registered */
ENTER; /* when it runs */
(VARS->enter_copy)(aTHX_ cb->ctx);
call_sv(cb->code, G_DISCARD | G_EVAL);
LEAVE;
(VARS->release)(aTHX_ cb->ctx); /* when it is dropped */
C code can also read and write any variable, or any element of an array or hash at any depth, by name. Nothing is linked against: the table is looked up once at load time and is versioned, so the two distributions upgrade independently.
What it costs
Measured on an Apple M5 with perl 5.42:
| cost | |
|---|---|
| read a variable | about 3x a package scalar, some 30 to 40 ns |
| assign one | about 6x a package scalar |
| call a sub that assigns once | under 2x the same sub using local
|
| a C loop's capture, enter and release per callback, nothing assigned | 4 to 5 ns |
| 100,000 captures of a context holding ten values | a fifth of the memory of 100,000 hash copies |
| read or write one element of a context hash | some hundreds of ns, against tens for a plain hash |
Scalars are cheap. Containers pay for being persistent and tied on every element access, so keep the large working data in ordinary lexicals and put in the context what genuinely belongs to it.
Getting it
Context::Vars 0.03 is on CPAN. It needs perl 5.10 or later, a C compiler and HAMT.
cpanm Context::Vars
Top comments (0)