Conda & Dealing with Conflicting Python(s) in your system

(Disclaimer: talking about a unix (me on linux) system here)

The problem

I took up a course on robotics using Robot Operating System (ROS). However, I always keep a miniconda3 installation to handle my data science adventures and respective dependencies. To clarify, it installs conda, a general purpose package manager - although really I only use it to install python libraries - and python 3.6.

What happens is that ROS uses the system python - i.e. python 2.7. The problem arises if you set your "python path" to default to that of miniconda3, a procedure which is automated when you accept that the miniconda3 installer adds the following line to your .bashrc file.

export PATH="/home/user/miniconda3/bin:$PATH"

In other words, everytime you open a shell, your system sets the python interpreter to that of miniconda3, which is version 3.6. So everytime I ran something from ROS, it would crash running python scripts that were written in version 2.7, namely due to syntax differences on the print command between the two versions.

The solution - meet symlinks

After reading conda's documentation on the subject, I decided to test my ability to implement their suggested solution, which is to:

• remove miniconda3 from the PATH
• create symlinks to three components of miniconda3
  • conda
  • activate
  • deactivate
• add these symlinks to the PATH

To remove miniconda3 from the PATH, one simply has to comment out or delete the aforementioned line in the .bashrc file, using some text editor like nano for example.

Now that that is done, let us create symlinks. A symlink, for the purpose of this post, is like a pointer to some other executable file and will function as a command. I created a specific folder (.symlinks) just for that.

mkdir .symlinks
cd .symlinks

Now that we're in the .symlinks folder, let's actually create symlinks.

ln -s /home/user/miniconda3/bin/conda conda
ln -s /home/user/miniconda3/bin/activate activate
ln -s /home/user/miniconda3/bin/deactivate deactivate

To verify that they were created, run ls -l (list files) in the .symlinks folder, you'll be able to see the pointers.

lrwxrwxrwx 1 user user 34 Nov  9 14:08 activate -> /home/user/miniconda3/bin/activate
lrwxrwxrwx 1 user user 31 Nov  9 14:10 conda -> /home/user/miniconda3/bin/conda
lrwxrwxrwx 1 user user 36 Nov  9 14:08 deactivate -> /home/user/miniconda3/bin/deactivate

At last, we add the .symlinks folder to the path. But we want to have these symlink commands automatically available every time we open a new shell. The answer is to "add" them to the script which is run everytime a shell is opened: the .bashrc file. So, add this line to the said file.

export PATH="/home/user/.symlinks:$PATH"

For the changes to take effect, you may close and reopen the terminal, but to feel like a pro without doing that move, run source .bashrc.

Now, if you run python --version you will get the system python back.

user@ryzen:~$ python --version
Python 2.7.12

If you want to use python 3.6 from the root conda environment, do

user@ryzen:~$ source activate root
(root) user@ryzen:~$ python --version
Python 3.6.3 :: Anaconda, Inc.

As you can see from the python --version bit, we're now using the 3.6 interpreter. If I want to activate the environment where I installed data science libraries¹ I'd run source activate ds.

Now that the newer interpreter is activated, you can run scripts that were written for that interpreter. Anyways, if you need to revert back to the system interpreter (2.7) on the current shell, then run

(root) user@ryzen:~$ source deactivate
user@ryzen:~$ python --version
Python 2.7.12

As you can see, we're back to 2.7. So you can run software that runs on this version with no problems, and switch back to your 3.6 antics whenever you want. 😁

Have a good one. 😁

1: I think conda does not allow to install packages to root anymore and forces users to create their own environments so as to incentivise good practices. Not that I checked, just personal experience.

Comments

[Aly Sivji]

I use conda for data science work; right now I have a simple bash function to start it up.

gogoconda ()
{
    export PATH="/Users/alysivji/anaconda/bin:$PATH"
    echo "Conda is activated"
}

Then $ gogoconda as required.

This seems a lot cleaner. Thanks!

[knzudgt]

@Aly Sivji, that is interesting. What script would contain your bash gogoconda function? Would you not get the same result by creating a 'gogoconda' shell script such as:

gogoconda.sh

#!/bin/bash

export PATH="/Users/alysivji/anaconda/bin:$PATH"
echo "Conda is activated"

[Aly Sivji]

I have that function in my ~/.bash_profile. And yes, you can also have a separate script. Might be easier to collect all your scripts into a central location.

[Rohan R Paleja]

For some reason, I am unable to use the activate, conda, or deactivate functions. It says no such file or directory. Do you know why this could be?

[Bernardo]

It sounds like conda is not properly installed. If it is, then the conda environment is not in $PATH. In linux, add the line export PATH="/home/your-username/miniconda3/bin:PATH" to your ~/.bashrc. Then retry.

[Rohan R Paleja]

I am using miniconda, so I am unsure if the commands are different. I am able to add that line, and then use commands like conda activate, etc. but I would like to jump back in forth between versions without having to change my bashrc everytime. I wanted to use this symlink method to fix that, but it doesn't allow me to use symlinks. Do I have to make them executable or change some internal setting?

[Bernardo]

sorry my bad!

• Miniconda is just a smaller version of Conda including just the bare minimum, such as the conda command

From what I can understand from what you said, I think you may have forgotten to add the symlink folder to the path!

So yes, scratch that thing that I told you in my previous comment (we were checking if conda was there).

Now, add this to the bashrc file (it always remains persistent to your last save and this file is run everytime you open a terminal) export PATH="/home/your-username/.symlinks:$PATH", and run source ~/.bashrc just to make sure that the path variable is updated (it's like re-running a script). Retry and let me know.

(you're on linux/unix right? I cannot help with Windows :))

[Rohan R Paleja]

Yeah I am running Ubuntu 16.04. So I did that and still same issue. I attached a picture. The only way to get it back to the miniconda location is to run unset PYTHONPATH and then when I run anything in a conda env, it finds the right path by itself.

[Rohan R Paleja]

Yes I am on Linux (specifically Ubuntu 16.04). I have added it to path and still getting the same error where the command doesn't register. Do you know any other common bugs it could be?

For now, I'm thinking to just add a question in the bashrc asking if I will be using conda during this session, and have that set the python path every time.

[Moritz]

Another (cleaner) possibility is to use conda without anaconda. That way you maintain your system's python environment completely and use conda as an environment manager with all the features you are used to.

pip install conda is all you need..

[Bernardo]

LOL so I could've skipped a whole hassle? Well good thing I learned about symlinks because of this haha. I always installed conda from the script, but I suppose I will start doing that.

[Danny Ayers]

Worked for me, thanks. I like your approach, no need to remember another script name or command.

[Bernardo]

yep