As a devoted Perl programmer, I find myself making heavy use of the fantastic Perl module repository, CPAN. And I do my best to keep up with all the latest goings-on with Perl.
I also have recently gotten hooked on conda, a neat tool which combines package management and environment management. If you haven't checked that out yet, I suggest you give it a look to see if it suits your needs.
TL;DR
(Full post after this tl;dr)
# envname is the name of your (existing) conda env
conda activate envname
# cpanm is available on either conda-forge or
# bioconda, so add conda-forge (or look up
# how to set up bioconda if you need tools
# for bioinformatics)
conda config --add channels conda-forge
# Install cpanm
conda install perl-app-cpanminus
# packagename is the name of the Perl package you want
env PERL5LIB="" PERL_LOCAL_LIB_ROOT="" PERL_MM_OPT="" PERL_MB_OPT="" $CONDA_PREFIX/bin/cpanm packagename
If that didn't work:
# Make sure gcc is installed
# Replace gcc with gxx if you need to compile C++
conda install -n envname gcc_impl_linux-64
conda activate envname
declare perlarchloc=$($CONDA_PREFIX/bin/perl -V::installarchlib: | sed "s/[' ]//g")
for conffile in "Config_heavy.pl" "CORE/config.h" "Config.pm"
do
sed -i -E 's%/tmp/build/[a-zA-Z0-9]+/perl_[0-9]+/_build_env%'"$CONDA_PREFIX"'%g' "$perlarchloc"/"$conffile"
done
unset perlarchloc
If that still didn't work, jump to the last section.
/tl;dr
Installing Perl and Python Packages
Conda is written in Python, and I think because of that there is a slight bias towards things in Python working well. Case in point, installing a python module which hasn't already been packaged in a conda channel is relatively simple:
# envname is the name of your (existing) conda env
conda activate envname
# Make sure pip is installed
conda install pip
# packagename is the name of the python package you want
pip install packagename
Cool. Now, how about a Perl package?
# envname is the name of your (existing) conda env
conda activate envname
# cpanm is available on either conda-forge or
# bioconda, so add conda-forge (or look up
# how to set up bioconda if you need tools
# for bioinformatics)
conda config --add channels conda-forge
# Install cpanm
conda install perl-app-cpanminus
# packagename is the name of the Perl package you want
cpanm packagename
If this worked for you, great! If it didn't, well let's see what could have gone wrong.
Problem 1: local::lib or other environment variables
A lot of us like to use local::lib for installing Perl packages locally, without needing to mess with system-level Perl. This works great in general. Not so much when you're letting conda take care of things.
There are a few reasons you want to use conda, and if you're reading this, you probably already know why (go ahead and look up more about conda if you really don't know how useful it can be). And as such, it's preferable for you to install your Perl modules for an environment only in that environment.
Maybe your environment uses a different version of perl than your system (or another environment). Maybe you share the conda environment with many users, and you want any user of it to have access to some specific module(s).
Unfortunately, having local::lib set up might mean that the cpanm command from earlier installed the module only for you, outside of the conda env:
$ perldoc -l packagename
$HOME/perl5/lib/perl5/packagename/packagename.pm
Ugh.
To avoid this, and any other environment variables that affect Perl modules from interfering, you might try this:
env PERL5LIB="" PERL_LOCAL_LIB_ROOT="" PERL_MM_OPT="" PERL_MB_OPT="" $CONDA_PREFIX/bin/cpanm packagename
The above worked for me, and its a bit ugly but I needed a solution in the moment. If you know of a better way, let me know in the comments and I'll update the post.
Problem 2: Modules using XS
It turns out that installing a module that uses XS (as many modules that need the speed of C, or access to Perl internals do) in this way does not work. It's a known bug in conda, too.
What happens is that the the configuration for cpanm retains some paths relative to the directory in which it was built, rather than to the conda environment (as you'll see in the last comment in that bug report).
That comment also explains how to fix the issue, but I will reproduce that solution here, a bit more generalized (this assumes the version of Perl you installed is 5.26.2, so make sure you check that first):
# Make sure gcc is installed
# Replace gcc with gxx if you need to compile C++
conda install -n envname gcc_impl_linux-64
conda activate envname
declare perlarchloc=$($CONDA_PREFIX/bin/perl -V::installarchlib: | sed "s/[' ]//g")
for conffile in "Config_heavy.pl" "CORE/config.h" "Config.pm"
do
sed -i -E 's%/tmp/build/[a-zA-Z0-9]+/perl_[0-9]+/_build_env%'"$CONDA_PREFIX"'%g' "$perlarchloc"/"$conffile"
done
unset perlarchloc
Now just rerun your cpanm command(s) and that should work! Unless...
Problem 2a: compilerroot?!
Turns out that on some configurations, a variable, $compilerroot, is used to set the location of the compiler in the files above. It's not easy to automate (for me), so I suggest you manually open the "Config_heavy.pl" and "Config.pm" files under "$CONDA_PREFIX"/lib/5.26.2/x86_64-linux-thread-multi/ and look for the following line:
my $compilerroot = "";
# Some other code is here
Change that assignment, and remove the if/else code following that line that conditionally sets $compilerroot too. Make sure you use the correct path for your environment's conda prefix (ie, the value of $CONDA_PREFIX when your environment is active):
my $compilerroot = "/your/conda/env/prefix";
# Don't change this code below
local *_ = \my $a;
Now try using cpanm :).
I hope this helped someone. It took me a few hours before I figured this out. In retrospect, it might have been easier to just package the missing perl modules for conda myself!
Edit: Feb 20, 2021. Updated the code block showing how to change the paths in the conda perl headers. Now instead of the perl version being hardcoded, use the location of the arch lib location given by the conda perl installation. Also updated post so that call to conda cpanm use the $CONDA_PREFIX environment variable.
Top comments (8)
Error:
Sorry for taking so long to get back to you, but it was because I wasn't able to reproduce this myself. However, I managed to run into it on another system. I believe that the reason why you're having this issue is that I actually advised installing the wrong packages for gcc/g++ in conda. They should actually be:
as these packages also add scripts which initialize some environment variables needed to tell the compiler where the conda installed toolchain is. It just so happened that for me, in the environments I was using, I already had these packages installed.
After installing these packages, deactivate/activate the conda environment (or start a new login shell somehow and activate the environment) and then please try again. If it works, I will update the post.
Edit: reference: docs.conda.io/projects/conda-build...
Hm, do you have gcc installed through conda?
Yes, I have installed
compiler.perl 5.26.2 h516909a_1006 conda-forge
I using this perl.
I fix this by:
conda install gcc_linux-64Thank you so much for this! Any ideas how to handle
$compilerrootin a more automated way? I was also thinking the real fix is to add functionality to github.com/conda/grayskull to create recipes automatically like it can do with pypi and cran