Most who have used python have been exposed to the reality of different python versions, different package versions, and the challenge of maintaining python versions for different project with different needs. This package for this project need python 3.3, while that core package for the other project still requires python 2.7.
While there are many ways that one can manage python environments, we suggest using conda (*not* anaconda), and when necessary, pip within conda. Conda manages more than python packages, it manages environments – you can install a particular version of python, and package, as well as R, and Fortran, if youy need to. Conda also makes it relatively easy to port environments between machines. So, if you need to get the python environment you’ve been running on your Mac laptop onto a linux compute node, Conda will provide tools for making that relatively painless.
If you have modifications to make to this document, see Scott Dynes or Ricky Leiserson.
Installing Conda
You will be installing Miniconda – not Anaconda. Miniconda is a package and environment manager; Anaconda is a data science stack installer that uses miniconda. We don’t want all that Anaconda brings in the way it is brought; so we will install miniconda and create our own environment.
See https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
and https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf
The install should update your PATH variable to include the conda path.
To create a python environment with conda:
conda create name <environment name> python=<python version>
Example to create an environment with python 3.8.1:
conda create --name conda_3.8.1 python=3.8.1
To activate this environment:
conda activate conda_3.8.1
To deactivate this (or any condo activated) environment:
conda deactivate
Note that the activation changes your prompt tag (on the left) to the name of the activated environment. The lack of any environment will have a tag ‘base’.
***Important*** You should *not* use virtualenv or venv in this conda environment, as conda is acting as the ‘virtualizer’. Although you will initially have no problem, they will likely set up their own directories outside of the conda directory, which could lead to portability issues.
To list the environments you have set up, run:
conda env list
To list the conda environment in a way it can be read in to set up another conda,
conda env export > ~/conda_requirements.yml
conda env export --from-history -n HERA_3.7.6 > ~/conda_requirements.yml
And to use that,
conda env create -f ~/conda_requirements.yml
Note that if you need to use a different version of e.g. python you will need to edit the file.
Set a pythonpath for .py files you’d like to include that are not part of python packages that are imported with pip or conda.