Setting up a Python development environment on Mac OS X

Right before the Christmas holiday, my new Mac arrived at work. This computer will be used mainly as a test and development environment for the software projects, and so needed a very flexible installation of Python. This took quite a while to get right, so here I will document the exact steps required to get it to this state. 

When developing software, people can discover all kinds of strange bugs that are related to particular software versions or computer platform combinations. For us to debug these, we need to be able to replicate any such combination. Therefore, I needed an installation which would allow me to select a given version of Python, and select particular versions of python modules to be combined with that (however if all you want to do is to install Python, the following is probably overkill).

Enabling this flexibility comes at the cost of increased complexity, but in the end I arrived at a decent workspace that allows the above pick-and-choosing of versions for any given Pythonic task. From this point on, this post will get a bit more technical than usual, but it remains well within the domain of the small-angle scatterer! That said, there are superfluous items in the following instructions, and there are probably ways to make this process more efficient, so feel free to leave comments with improvements!

For the following, all lines following the “$” (string) character are to be typed into a terminal window (which you get when opening the “terminal” app). The characters “~/” denote your user home directory, in my case this is the directory “/Users/brian”. While I understand from some (anonymous) beamline managers that some (anonymous) beamline users may be a bit hesitant to use any text-based interface, I do recommend playing around with these a bit. After a while, you will find it easy to be able to do some tasks in there which can be replicated every single time just by copy-pasting some commands. Anyway, without further ado, the installation guidelines for a fresh install of OS X 10.11:

• Install some “essential” OS X software, in this order: GPG Suite, OpenSCAD, SourceTree, Tor Browser, mactex, Xcode (the latter via the App store, currently version 7.2, build 7C68).
• modify (or create) ~/.bash_profile to include the following text:

export LC_ALL=en_GB.UTF-8
export LAN=en_GB.UTF-8
export ARCHFLAGS="-arch x86_64"
# export PATH=/usr/local/bin:$PATH # (no longer necessary, already correct)
test -f ~/.bashrc && source ~/.bashrc

• modify (or create) ~/.bashrc to include the following text:

export PIP_REQUIRE_VIRTUALENV=true
# define a "global pip" function to use outside virtualenv:
gpip(){
    PIP_REQUIRE_VIRTUALENV="" pip "$@"
}
test -f /usr/local/bin/virtualenvwrapper.sh && source /usr/local/bin/virtualenvwrapper.sh
if which pyenv > /dev/null ; then eval "$(pyenv init -)"; fi

• Update the running terminals with the new settings using:

$ source ~/.bash_profile

• Install Homebrew for OSX software package management, and configure:

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew doctor
$ brew tap homebrew/science
$ brew tap homebrew/python
$ brew tap homebrew/versions
$ brew update
$ brew upgrade

• Install Pyenv which allows the installation of isolated versions of Python

$ brew install pyenv

• Install the Xcode command-line tools (click the highlighted option on the dialog that pops up):

$ xcode-select --install

• Install a Python environment before we can continue (2.7.11 is the most modern python2 version):

$ pyenv install --list
$ pyenv install 2.7.11 
$ pyenv local 2.7.11

• Now we can install virtualenv, which allows us to separate the python module versions:

$ gpip install virtualenv

• And the package that manages the mix of python versions with the mix of module versions (note: don’t do the “auto-activate” thing):

$ brew install pyenv-virtualenv

• Now we can set up a custom environment ideal for running McSAS. First we install the required software for the graphics back-end (pyside 1.2.2 (we need the post install script which has been removed from later versions of pyside)), and then we set up a modern python2 environment (2.7.11) called “mcsas”.

$ brew install cmake qt mercurial pandoc
$ brew linkapps
$ pyenv virtualenv 2.7.11 mcsas
$ pyenv activate mcsas
$ pip install numpy scipy matplotlib
$ pip install -Iv pyside==1.2.2 
$ python ~/.pyenv/versions/mcsas/bin/pyside_postinstall.py -install

• You can check the module versions that are installed for each of the modules using:

$ pip freeze

• which returns:

cycler==0.9.0
matplotlib==1.5.0
numpy==1.10.2
pyparsing==2.0.7
PySide==1.2.2
python-dateutil==2.4.2
pytz==2015.7
scipy==0.16.1
six==1.10.0
wheel==0.24.0

• You see that there are several modules installed that we did not explicitly install ourselves. These are installed automatically when you install a given module, because that module needs it. (This list can be stored in a text file for easy reinstallation later on if desired). We can now set up a separate Python environment for running python (jupyter) notebooks:

$ pyenv virtualenv 2.7.11 notebook2
$ pyenv activate notebook2
$ pip install scipy numpy matplotlib pandas ipython jupyter h5py 
$ ipython notebook

• 

  Or a notebook using python 3 (see Figure 2):

$ pyenv install 3.5.1
$ pyenv virtualenv 3.5.1 notebook3
$ pyenv activate notebook3
$ pip install scipy numpy matplotlib pandas ipython jupyter h5py
$ ipython notebook

In all of these cases, each python installation and module configuration lives separate from each other, which makes reinstallation, upgrading and testing very easy. As mentioned, this is overkill if all you want to do is to install a single Python, but it is very useful for making sure that every program gets exactly the environment it likes best.